Building Consensus via a Semantic Web CollaborativeSpace

George AnadiotisIMC TechnologiesAthens, Greece

ganadiotis@imc.com.gr

Konstantinos KafentzisIMC TechnologiesAthens, Greece

kkafentzis@imc.com.gr

John PavlopoulosIMC TechnologiesAthens, Greece

ipavlopoulos@imc.com.grAthens University of

Economics and BusinessAthens, Greece

annis@aueb.grAdam Westerski

Universiadad Politecnica deMadrid

Madrid, Spainwesterski@dit.upm.es

ABSTRACTIn this paper we outline the design and implementation ofthe eDialogos Consensus process and platform to supportwide-scale collaborative decision making. We present thedesign space and choices made and perform a conceptualalignement of the domains this space entails, based on theuse of the eDialogos Consensus ontology as a crystalliza-tion point for platform design and implementation as wellas interoperability with existing solutions. We also present ametric for calculating agreement on the issues under debatein the platform, incorporating argumentation structure anduser feedback.

Categories and Subject DescriptorsH.4.2 [Information Systems]: Types of SystemsDecisionsupport; H.5.3 [Information Systems]: Information Inter-faces and PresentationGroup and Organization Interfaces;J.4 [Computer Applications]: Social and Behavioral Sci-ences

General TermsComputer Supported Collaborative Argumentation

KeywordsCollaborative Decision Making, Issue Based Information Sys-tems, Argument Mapping, Argumentation, Citizen Engage-ment, Consensus, Alignment, Process, Platform, Metrics

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.SWCS 2012 Lyon, FranceCopyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00.

1. INTRODUCTIONIn recent work, we have examined Virtual Communities

as emergent socio-technological phenomena and shown howthe increasing level of connectedness among people via theuse of ICT -what is termed Social Media (SM)- has both cat-alyzed and been catalyzed by this tendency. Furthermore,we have examined how SM empower Virtual Communities,how Semantics empower SM as knowledge elicitation plat-forms leading towards the Enterprise2.0 and Government2.0paradigms and how we can leverage these phenomena viaOpen Innovation to build a structured deliberation processthat goes beyond open, towards inclusive policy making [3].

In this paper, we present the eDialogos Consensus Build-ing platform, an initial implementation of the outlined OpenInnovation structured deliberation process: social semanticsoftware with the mission to bring together human agentsand software agents in order to foster knowledge-intensivecollaboration via content creation and management. In otherwords, a Semantic Web Collaborative Space with the ulti-mate goal of promoting collective decision making.

The rest of the paper is structured as follows: we beginby presenting the background and rationale for the designof the eDialogos Consensus Building platform in Section 2,and present the platform and its functionality in Section 3.In the following sections, we focus on the unique aspects ofthe platform, namely the Open Innovation Semantic Webinfrastructure for collaborative decision making (Section 4)and the Consensus Building Model (Section 5). Finally, weconclude and present future work in Section 6.

2. BACKGROUND AND RELATED WORKIn order to trace the roots of ICT-supported collaborative

problem solving, we have to revisit the IBIS methodology[11]. As IBIS was one of the first approaches designed ex-plicitly with the goal of being applied using information sys-tems, inevitably all subsequent collaborative problem solv-ing approaches somehow reference it. IBIS’ success lies inthe fact that it provides a structure and a process simpleto conceive and use, and yet powerful enough to express

argumentation cycles in groups. Initially, the general areain which issues exist is identified. Within this area issuesare identified through discussion, and each is recorded as aquestion. Possible answers to the question are identified aspositions and then participants express arguments in favourof or against a position. Unravelling issues in this fashionmay lead to an answer on which all participants agree, inwhich case the process terminates, or if this is not the casethen the positions and arguments are re-examined -whereverpossible treated as issues themselves- and the decompositioncycle repeated.

The system was originally described in terms of a modeland algorithms, although at the time of writing the proce-dures were implemented largely by hand. This was of coursea major shortcoming of IBIS, but in the following years anumber of tools to support the process have been developed.This development has taken on a couple of defining charac-teristics, namely formal argumentation and argument map-ping, that lead to the creation of a new tool genre. This toolgenre is an offsrping of gIBIS, a sophisticated, hypertext-based graphical tool for using the IBIS techniques, whichlater led to the development of the commercial QuestMaptool, in turn superceded by Compendium [12]. Other toolsin this genre are the recently introduced PolicyCommons [7],based on the Cohere tool [26], and the Carneades tool in itslatest incarnation [14].

The aforementioned tool genre is used mainly in the do-main known as ICT for Governance and Policy Modelling(IGPM). IGPM is traditionally thought of as being com-prised of two complementary yet separated research fields:the Governance and Participation Toolbox, including tech-nologies for mass conversation and collaboration (Social Net-working, Web2.0/3.0 and Crowdsourcing approaches) andthe Policy Modelling domain including technologies such asProcess Modelling, Visualization, Gaming, Mixed Realityand Simulation [1]. This divide is evident in the above men-tioned tools and their characteristics. Since formal argu-mentation is approached based on complex logic contructs,suitable for legal argumentation but hardly approachable foraverage users [29], while argument mapping is based on GUIelements for interactions, most tools tend to:

• Focus on the argument mapping aspect, making fortools that are user-friendly but offer limited function-ality - glorified mind maps.

• Focus on the argumentation grounding aspect, makingfor a complex and unappealing user experience - madeby and for argumentation experts.

So in terms of striking a balance between expressivity andreal-world applicability, it seems that the Cope It tool [16]with its (limited) support for import from existing forums,the Deliberatorium tool [17] aiming at large scale argumen-tation and the widely acclaimed DebateGraph 1 tool are asclose as it gets at this point. For an in-depth review andanalysis of the domain, the interested reader may refer to[25].

An obvious shortcoming that spans these tools is lack ofinteroperability: although the overlap in shared process andconcept space is striking [13], it’s not until recently that theinteroperability effort made its first steps [22], covering only

1http://www.debategraph.org

the syntactic aspect while ignoring the semantic one. A pro-posal in this direction has been put forth [5], but it seemsto remain dormant and is not used in any of the tools. Thisis rather surprising, if we consider that ”IBIS in effect cov-ers knowledge elicitation, modelling and reasoning. The useof atomic statements and relationships between these state-ments anticipates some of the core techniques of SemanticWeb developers three decades later” [5].

Another related effort for semantic grounding and interop-erability is expressing the AIF argument exchange format asan ontology [23], in support of a vision to lay the foundationsfor a World Wide Argument Web that remains unfulfilledup to this day. In addition, by following the AIF formatthe resulting ontology is also built mainly to support theexpression of argumentation schemes, resulting in a rathercomplex schema. There is also the SWAN ontology, whichincludes elements of scientific discourse and argumentation.Even though much simpler than its AIF-based counterpartthough, this ontology still takes a formal approach to argu-mentation. This results in a schema appropriate to supportscientific research with hypotheses and supporting evidence(this was its intended use in fact), but does not lend itselfvery well to representing argumentation on wicked problemsby a large number of stakeholders [10]. Finally, OPOL ex-presses the representation of political programs as an ontol-ogy [6], which while being useful in its own right does not sayanything about the argumentation and process of decisionmaking.

A crucial aspect missing from the aforementioned toolsis support for incorporating user feedback in the decisionmaking process. The assumption seems to be that layingdown the arguments and/or providing a structured processwill facilitate the decision making in one of two ways:

• By resulting in an enlightened understanding of theproblem on behalf of the participants, who will some-how reach a decision via offline procedures - typicallyrelying on the role of the facilitator, an individualskilled in the process, the tools and able to commu-nicate effectively with all participants.

• By documenting all the arguments, their logical premisesand structure, so that applying reasoning rules will en-able tools to provide the ’algorithmically optimal’ so-lution.

Unfortunately, neither assumption holds when applied towicked problems with a big and diverse group of participantsand stakeholders, as most real-life public sphere issues tendto be.

While evidence shows that documenting arguments in astructured way is a very effective knowledge elicitation mech-anism [17], and domain knowledge certainly promotes deci-sion making capabilities, everyone cannot be an expert inevery domain. But even if they could, this alone would notmake for an ideal decision making situation (cf [15]), or pro-vide the means to capture the input required for decisionmaking. Relying solely on formal argumentation and mod-elling on the other hand does not resonate well with stake-holders, as the underlying modelling typically used does nothave a history of sucess that would make it trustworthy andis also quite different from the behavior observed in humansocieties [1]. Therefore, it promotes the already existingfeeling of disconnectedness from the public sphere.

An approach that tries to address this is Citizen Engage-ment (CE). CE builds on both ICT and political scienceand bears the promise of reconnecting citizens and all otherstakeholders to decision-making and governance, central orlocal, by means of policy formulation and evaluation. Theultimate goal however is to strengthen and empower demo-cratic governance at all levels of society and to promote in-dividual citizen evolution as part of the collective process ina deliberative, discursive way. Ironically though, all tech-nical and legislative issues aside, what remains the greatestchallenge in today’s CE landscape is actually achieving sub-stantial CE levels [21].

Since at its foundation, SM is a set of technologies andchannels aimed at forming and enabling a potentially mas-sive community of participants to productively collaborate,it seems that CE and SM have common ground and a wayforward for CE could be to learn from and adopt the SMparadigm. Experience with such systems has shown thatthey can foster, by virtue of reducing the cost of participa-tion, voluntary contributions at a vast scale, which in turncan lead to remarkably powerful emergent phenomena, suchas idea synergy, the long tail, many eyes and wisdom of thecrowds [17].

These emergent phenomena manifest themeselves in emer-gent communities. Traditionally, CE efforts are associatedwith governmental organizations, as they are the ones withthe means and the legitimacy to sponsor such projects. Latelythe increasing need for self-governance in emergent com-munities, potentially geographically dispersed and typicallycommunicating over SM, has given rise to a new genre ofdecision-making approaches. These approaches range fromthe simple ’outsourcing’ of functions of traditional politics,such as campaigning [8], to a SM platform, through the veryuse of SM capabilities in ways that redefine the decision mak-ing process itself, such as expertise and trust-based vote del-egation [27, 24], grass-roots decision making infrastructure[2], or open innovation deliberation [3].

However, while SM practices may indeed provide bene-fits, since existing SM are not designed for CE they sufferfrom a number of drawbacks that render them inappropriatefor this purpose, namely disorganized content, low signal-to-noise ratio, quantity rather than depth, polarization anddysfunctional argumentation [17]. What is needed there-fore is a new genre of tools that will stand in the middleground between completely unstructured, general purposeSM and highly structured, formal argumentation tools. Thegoal would be to harness the power of user-generated con-tent, feedback and interaction via a process and platformthat provides only as much structure as necessary, withoutbeing a burden to learn and use.

In this respect, the use of metrics to estimate exceptionshas been proposed: exceptions in this context are situationsthat potentially require user intervention or are interestingfor users [17]. The approach tries to identify these excep-tional situations qualitatively and then model them as ametric that expresses the expression quantitativaly, basedon available data. The rationale for this kind of metrics,introduced in [17], is to direct user attention to topics andevents best-suited for them, picked from a potentially vastcollaboration space. However, no concrete metrics are pre-sented. In [19], a metric for evaluation of argument strenghtis introduced, however it is purely theoretical (based ongame-theoretical modeling) and based on the assumption

that different arguments on a topic could somehow be as-signed a strenght factor through some unidentified process.Finally, in recent work [18] the notion of Social AbstractArgumentation is introduced, defining a framework to takeuser feedback via SM into account when calculating argu-ment strength.

3. A DELIBERATIVE, DISCURSIVE MODEOF DECISION MAKING: THE EDIALO-GOS CONSENSUS PROCESS AND PLAT-FORM

Based on the domain analysis of Section 2, we have em-barked to design a process and a platform for a deliberative,discursive mode of decision making that incorporates best-of-breed characteristics and tools from each discipline andapproach. Our work has been inspired by existing method-ological and technical background, while at the same timeincorporating experience gained through application of de-liberative CE processes in a number of real-world settingssuch as local 2 and regional 3 government or mainstreampolitical parties 4. We documented a theoretical frameworkand established a structured deliberation process [3], utiliz-ing Semantic Web Technology to facilitate dialogue [4] andthe Open Innovation principles [9].

Figure 1: The Open Innovation deliberation process

Our priorities in this process were:

• To make the entry barrier for users as low aspossible. This lead us to design the system as a SMplatform to make for an intuitive user experience, in-corporating well-known tools such as one-click contentcreation and reaction facilities, groups for sharing andcollaborating with others on specific topics, activitystreams for keeping up to date with user actions andreactions and different ways of organizing, navigatingand locating content and users, such as hierarchicalstructure, horizontal association and universal search.

• To build on and maintain compatibility with ex-isting approaches. This lead us to adopt the IBISmethodology, as the one that underpins a wide arrayof existing approaches and is well known, well under-stood and uses a simple yet powerful notation thatenables the expression of collective decision making asa graph that can be documented and explored. Fur-thermore, in order to align approaches and systems,we provide a way to conceptually document alignmentas well as facilitate interoperability on the technicallevel, by means of an ontology.

2http://epractice.eu/en/cases/edialogosawards3http://www.samos-dialogos.gr4http://www.nd.gr/web/dialogos

• To enable, encourage and make use of user-generated content and feedback in every phaseof the process. Using tools that facilitate the me-chanics of user engagement and pairing them with adesign that makes for an intuitive user experience is aprerequisite to achieve this goal. Going one level be-yond this however means designing and implementinga model that estimates argument strength and agree-ment level based on user feedback. Furthermore, amodel of participation as a strategic game that rewardscontributions and engagement would further promotethe process.

The opportunity to apply this process in practice and de-ploy the platform presented itself when we were contacted bythe European Academy of Allergy and Clinical Immunology(EAACI5), with the aim of implementing a process throughwhich its 7000 members would be enabled to collaborate ina distributed fashion in order to collectively author medicalguidelines. In order to sucessfully apply the process in thecontext of decision making taking place in EAACI medicalguidelines formation, we had to consult with EAACI mem-bers and adjust the process as needed. The Open InnovationeDeliberation process is a tight ’serial process’ within a spe-cific time-frame, with 6 discrete phases embedded in eachdeliberative cycle, depicted in Fig 1.

Figure 2: The eDialogos Consensus deliberation pro-cess

Initially we have the agenda setting phase, in which par-ticipants may choose among a list of proposed topics fordeliberation, or propose their own topic. After having cho-sen a general topic for the deliberation in the agenda set-ting phase, we break down this topic in specific problems toaddress. Problems must be clearly formulated in order tobe addressed, and this process is supported by appropriatetools. The outcome of the problem formulation phase will bea number of problems, therefore an idea management phasefollows in the process in order to collect ideas for dealingwith each problem. After the idea management phase con-cludes, a proposal shaping phase is initiated, with the aimof composing and refining ideas into well-rounded proposalsfor adoption by the hosting organization. Finally, the vot-ing phase on completed proposals gives participant input forthe decision making that takes place in the Online Councilphase, determining proposal adoption, implementation anddeployment.

In the case of EAACI, the agenda setting phase was omit-ted. This phase is essential when the decision-making do-main is very broad, as is the case for political decision mak-ing, but for a community such as EAACI, operating withina specific domain, it was preferred to simplify the process.It must be noted however that there is the notion of Task

5http://www.eaaci.net

Forces, which can be considered to implicitly contain theconcept of topic. Task Forces are groups of individual usersand are meant to accommodate special interest groups withinEAACI.

The problem formulation phase however corresponded di-rectly to the EAACI guideline decision making process, asit involves formulating a specific issue for which a guidelineneeds to be authored. Adopting our terminology to IBIS, wechose to refer to problems as Issues in this context. Issuesare formulated and shared within a Task Force by the TaskForce moderators (using a simple form web interface for thispurpose) and remain open for input for a predefined period.This open for input period is broken down to a debating pe-riod (idea management phase) and a voting period (votingphase).

The idea management phase was also a natural match tothe EAACI guideline decision making process, as for eachformulated issue members may either create their own po-sitions (limited in number in order to prevent abuse), orparticipate by engaging in conversation with other mem-bers, commenting and stating opinions on the advantagesand disadvantages of existing positions. In order to take ad-vantage of increasing user familiarity with SM, we chose toutilize a set of well-known SM features for collecting feed-back, aligned with the IBIS philosophy: simple, yet capableof capturing rich dialogue semantics and structure. Usersare able to give feedback in one of the following ways:

1. Add note. Notes are meant to provide additionalinformation in the context of what is being discussed,and can be attached to positions, other notes, as wellas arguments for/against.

2. Add argument for. Arguments for are meant toprovide additional support for something that has beensaid in the discussion, and can be attached to positions,notes, as well as other arguments for/against.

3. Add argument against. Arguments against are meantto provide justification for refuting something that hasbeen said in the discussion, and can be attached to po-sitions, notes, as well as other arguments for/against.

4. Rate. Ratings are meant to enable users to take astance for or against something that has been said inthe discussion, and can be applied to notes and argu-ments for/against.

5. Vote. Votes are meant to provide evaluation of can-didate positions using a range voting system6.

The voting phase has been shifted in the EAACI guidelinedecision making process as follows: while an issue remainsopen for debate, task force members may create positions,add notes and arguments for/agains and rate - all part ofthe idea management phase. Voting on positions takes placeafter the debating period ends, during the voting period.During the voting phase users can vote on as many positionsas they like, giving each a score in a predefined range (0-10).Users can only vote once for each position, altough they arefree to change their vote as many times as they wish for theduration of the voting period.

6http://www.rangevoting.org

Figure 3: Issue view

This shifting also reflects the fact that the proposal writingprocess is currently out of scope of the EAACI guideline de-cision making process, so the voting takes place after the lastphase where user input is available. The proposal composi-tion phase will be introduced in the process and the platformas well, after a startup period during which platform userswill become accustomed to the basic decision making oper-ation. The goal is to semi-automatically compose proposalsbased on the highest-rated positions and then validate themvia a brief feedback and voting phase as well.

The resulting process is depicted in Fig. 2. In Fig. 3we show a screenshot from the platform, in which an Issueview during the debating period is displayed. We can see thedescription of the Issue, as well as additional information,Issue references, the Issue activity stream, the positions ex-pressed for it as well as the argumentation graph for one ofthese positions.

In order to model our domain appropriately before em-barking on platform design and development, the theoreti-cal background of Section 2 was considered and a concep-tual alignement was performed, resulting in the definition ofthe domain and its main concepts. These were captured bymeans of a domain ontology (see Section 4), which was usedas the focal point of an ontological model driven architec-ture. The platform was developed using standard web appli-cation architecture and tools, so there is a data storage layer

for which a relational database is used, an object-relationalmapping layer that abstracts the storage layer, an applica-tion logic layer that exposes Service APIs and a GUI layerthat uses these APIs to build the front-end of the platform.

Additionally however, a mapping of the data storage layerto the domain ontology definition was performed, so that de-liberation data can be exposed in a semantically well-definedformat, as per the Linked Data principles. The mapping wasperformed using the D2R tool, providing an interoperabilitylayer that enables semantic interoperability with other ap-plications in the domain: deliberation-related data can beexchanged directly via a SPARQL endpoint.

Furthermore, the conceptual model was used as the basisfor the design and implementation of a model that estimatesargument strength and agreement level based on user feed-back: the consensus rate model. As the crystallization of theconceptual model was the starting point for our design, thisenabled us to work in parallel on the platform implemen-tation and the the consensus rate model design, integratingit in the platform as the final step of its development. Theconsensus rate model has been implemented as a distinctmodule in the platform application depicted in Fig. 4, ex-posed via the Service Layer.

Figure 4: Consensus platform architecture

4. SEMANTIC WEB TECHNOLOGY TO FA-CILITATE COLLABORATIVE DECISIONMAKING: THE EDIALOGOS CONSEN-SUS ONTOLOGY

Following the definition of the platform architecture, oneof the requirements for the deployment of the system atEAACI was to ensure interoperability on data level withother deliberation models (see Section 2), including deliber-ations as modelled in the previous version of the eDialogosplatform. In addition, the data had to be easily publish-able and portable to allow tool-independent analysis. Dueto the intensive use of Web technologies in the implemen-tation, we have chosen to design the entire data back-endfollowing Semantic Web data modelling principals.

To the best of our knowledge, currently available for-malizations of deliberation models do not provide a singleontology that would cover all the concepts and mechanicsthat were required for the Consensus platform. Therefore,we have created an ontology that integrates contemporaryachievements in the domains of Problem Modelling, IdeaManagement and Collective Decision Making. In the follow-ing we elaborate on the manner in which the correspondingontologies (see Table 1) were integrated. More specifically,we describe the new concepts that the Consensus ontologyintroduces in order to facilitate the interoperability betweenthe domains of Idea Marketplaces, Idea Management Sys-tems and Issue-Based Information Systems.

To establish the core of the Consensus platform and facil-itate the aforementioned integration between different mod-

els, the eDialogos Consensus ontology defines three mainclasses: Argument Map, Issue and Position. Additionallyto represent discussions over ideas we introduced: Note andArgument concepts. Each of those classes inherit from anumber of classes of external ontologies that focus on mod-elling respective concepts (see Fig. 5):

• consensus:Issue is primarily based on ProblemChal-lenge concept that allows definition of various problemtypes and statuses. In addition since we recognizedIdeaContests as a similar concept on the Idea Manage-ment side, inheritance from both allows smooth reuseof the Gi2MO ontology properties to integrate bothmodels. Issue also inherits from the IBIS ontologyQuestion concept, as it is the starting point for thedecision making process.

• consensus:Position is based on the Idea concept fromthe Gi2MO ontology. Positions also extend ibis:Idea,as they correspond directly to this concept. Having aconcept that inherits from both aforementioned onesensures a consistent modeling that maintains links withboth approaches, while clearly separating from the some-what unintuitive modeling of the IBIS ontology thatbases its whole concept hierarchy on the Idea concept.The new class enables categorization using SKOS con-trolled vocabularies, as well as free-text annoatationusing tags, expressed via the SCOT ontology. It alsoallows direct relationships with ArgumentMaps thatdo not have their counterpart in Gi2MO ontology.

• consensus:ArgumentMap expresses the concept of

Table 1: Consensus ontology importsOntology Description of concepts modelledDcterms Generic properties for many assets, e.g. ’title’, ’description’ etc.Foaf Relation between User Account in the deliberation platform and personal dataScot Tags and tagging activitiesProblem Challenge Ontology Problem concept, its changes over time and challenge marketplaces modelIBIS Ontology Basic deliberation cycle model, relationships between ideas and associated discussionseDeliberation Ontology eDeliberation conceptGi2MO Ontology Idea, Idea Contest, concepts related to expert and collaborative review process

Figure 5: Integration of various ontologies for the eDialogos Consensus model

a common space for sharing all content generated asa by-product of a deliberation process, as described inthe eDialogos deliberation ontology [4]. This conceptalso corresponds directly to the Map concept of theIBIS ontology, shared by all argument mapping tools.In addition, to link it with the problem concept inIdea Marketplaces we model the meaning of Argumentmaps as an extension of ChallengeMarketplaces fromthe ProblemChallenge ontology

• consensus:Note and consensus:Argument - bothof those concepts relate to the deliberation model pre-sented by IBIS. Notes are the base concept for Ar-guments, since they both have content and are at-tached to Positions and Notes and Arguments (usingthe ibis:refers to and ibis:pro/ibis:con properties forNotes and Arguments respectively). In addition, toconnect those concepts to the Idea Life Cycle of IdeaManagement Systems (and the gi2mo:Idea class) weinterpret this part of the IBIS workflow as the IdeaSelection stage of the Idea Life Cycle [28] which is rep-resented in Gi2MO ontology as Idea Review.

On top of the above relations in the eDialogos Consensusmodel we extensively reuse elements of the Gi2MO ontologyto represent: gi2mo:Users as creators of any of above con-cepts, gi2mo:UserGroups to represent Task Forces deliberat-ing around selected topics aggregated with ArgumentMaps.

The categorization and organization of data is modelled us-ing the skos:Concept and scot:Tag formalizations for con-trolled vocabularies and free-text annotation respectively.

The ontological infrastructure of the eDialogos Consen-sus platform served a number of purposes. In addition todriving the Model-Driven Architecture for the platform andserving as the interoperability layer with other solutions inthe domain, in parallel it facilitated the development of theargumentation and feedback based consensus rate estima-tion model outlined in the following section.

5. ARGUMENTATION GRAPHS AND USERFEEDBACK TO ESTIMATE AGREEMENT:THE CONSENSUS RATE MODEL

One of the design goals for the Consensus platform waspragmatic, functional support for argumentation and de-bate, in a way that is easy to use for the average web user,and yet can provide tangible benefits in the decision mak-ing process. Following the conceptual design of the domain,we incorporated support for argument graphs (via the con-sensus:Argument concept and the ibis:for and ibis:con prop-erties) that can be attached to Positions as well as otherArguments. We also support the incorporation of user feed-back on these Arguments in the form of thumbs up/thumbsdown instant reactions. The crystallization of the domain asan ontological model was the basis that enabled us to work

in parallel on application and model development.Our goal was not to develop a model for the detection of

exceptional content/events, but rather to combine argumentgraph structure and user feedback to calculate a consensusrate that expresses the agreement level for topics under de-bate. This is orthogonal to the exception detection logic of[17], which in our case is partially supported by the activitystream updates generated for each user. The model was de-veloped through a process that combined intuition from ourdomain experience and experience documented in relatedwork. Arguments are viewed as nodes in a graph formed bytheir relation to each other, and for each node in this grapha consensus rate expressing public degree of agreement iscalculated, based on node feedback (for leaf nodes) as wellas the propagated consensus rate of the nodes connected toit in a recursive fashion (for parent nodes). In the follow-ing we outline the consensus rate model through a series ofexamples.

Figure 6: Baseline consensus rate calculation

We start by defining the consensus rate for leaf nodes. InFig. 6, we have a parent node A with two leaf nodes, Band C. Given that node C (similar for node B) is an ar-gument for or against node A, users are able to agree ordisagree with the argument by rating it with thumbs up orthumbs down respectively. We hereby describe a recursivemodel over the height of the argumentation tree. Initially,we compute absolute agreement of an argument in the leaveslevel which will be the base case of the recursion. That is,given a parent node A that is a thesis (argument support orattack) and two children arguments B, C (again, either insupport or attack) we first calculate the agreement level foreach node. This calculation is based on the thumbs up anddown votes for each node and is expressed as follows, as therate of absolute agreement:

rabs(m) =

∑v∈votes(m) v∑v∈votes(m) 1

(1)

The rate of absolute agreement of a node is the ratio ofthe summary of agreement, to the number of votes. Thisranges from -1 to +1, where values greater than 0 indicateacceptance of the argument while values less than 0 indi-cate dissaproval. For example, if 23 people disagree and 5agree with some argument, we get an absolute agreementof −18

28= −.0.64. This score is characterized as absolute

due to the local nature of the denominator. This is moreclearly explained in Fig. 6, where we assume that nodes B& C are populated with thumbs up and down votes. Giventhat one person agrees and two disagree with B, we get arate of -0.33. On the other hand, if 300 people agree and400 people disagree with C, we get a rate of -0.14, which issignificantly lesser. Intuitively, we would expect to have agreater rate for C, as in this case we have more user input

and therefore the public opinion extracted by this is morereliable. Therefore, this a rather misleading result, in thesense that it does not take into account contribution to thecomment voting. This missing factor is defined as a secondmetric, the Relative Rate of Agreement:

rrel(m) = rabs(m) ∗∑

v∈votes(m) 1∑k∈nodes(Issue)

∑v∈votes(k) 1

(2)

Relative Rate of Agreement for some node m is defined tobe the rate of absolute agreement for m, multiplied by theparticipation in that node. Participation normalizes over allnode descendants of an Issue (all the arguments in this Issue)and thus provides a more relative perspective. Applying inthe previous example where B had 3 votes and B had 700,the relative agreement is computed as:

rrelative(B) = −13

3703

= −0.0014, and

rrelative(C) = −100700

700703

= −0.14That is, the contribution factor degrades the rate of node

B while leaving almost intact the rate of node C. The relativerate of an argument measures how much do people agreewith it (thumbs up and down), out of all the people whocould have agreed and it ranges from -1 to 1, the same rangeas the absolute rate. Such a measure is important in orderto promote arguments that generate lots of feedback. Thisscore may be used recursively upwards the tree, to measurethe rate of parent arguments. In the following we define therecursive rate of a node m, based on the hypothesis that allof node children are already recursively rated:

rrec(m) = a1rrel(m)+a2

∑c∈children(m) rrec(c)(rrec(c)− e)2

b +∑

c∈children(m) 1

(3)Variables a1 and a2 are weights defined manually. The

sign of both weights reflects the type of the argument. Thatis, a1, a2 > 0 if argument is for and a1, a2 < 0 else. Variablea1 acts as a weight of the relative agreement in the nodedefined in (2) while variable a2, weights the rate comingfrom the descendant subtrees.

Figure 7: Degradation of argument strenght fordeeply rooted arguments by colluding users

The rate from the descendants is defined to be the sum ofchildren rates, scaled by how much each rate deviates from

the mean∑

c∈children(m) rrec(c)∑children(m) 1

. In our case, this scaling fac-

tor awards scores which deviate more than normal as these

should be relatively important. The outcome is normalizedwith the number of children and smoothened by a variableb. This variable only affects the case of deeply rooted argu-ments with single parent nodes (see Fig. 7). If this is zero,the root of a chain sub-tree created by single parent argu-ments (left child of node m) receives a high value on thesecond factor of the right part of the equation. This meansthat, deeply rooted and highly evaluated arguments are im-portant. Else, if this is non zero, this factor degrades to itsway up. This is useful to avoid the effect of colluding users.Such a user could create o chain of mutually reinforcing ar-guments for, in order to promote a root node. This is shownin Fig. 7, in which the shade of green for each node in thechain represents the recursive rate given to the node accord-ing to the convention that a darker shade of green stands fora higher rate. Since a chain like this one would not receivea high relative rate (assuming that such a scheme would notbe supported by a substantial percentage of users), the lastnode is colored in light green. However, while going up, thesmoothing parameter acts as a control in order to penalizethe chain argumentation.

6. CONCLUSIONS AND OUTLOOKWe have presented the eDialogos Consensus process and

platform, as derived from our own field experience and pre-vious work in the domain of deliberation and collective deci-sion making as well as other existing approaches. Our goalwas to develop a process and platform that is easy to useand pragmatic, while also incorporating features that try toaddress gaps in the applicability of existing approaches. Atthe same time we wish to adopt these features of existingapproaches that have shown potential and maintain compat-ibility with them, offering an aproach that can also act as abridge for existing ones. We have tried to achieve these de-sign goals via a modular design and user-friendly interface,adoption of a common methodology and its concepts, andour two contributions to the field:

• The development of the infrastructure for a conceptualand technical bridge among various decision makingsystems, by means of the eDialogos Consensus ontol-ogy.

• The development of a metric that combines argumen-tation structure with user feedback to provide an in-dication on the level of agreement and the strength ofarguments on topics under debate, by means of theeDialogos Consensus rate.

The result has been implemented and is about to be de-ployed in order to constitute the decision making and de-liberation backbone for the EAACI medical association. Aninitial beta testing phase has been rolled out in mid-Februaryof 2012 and is available at http://consensus.imc.com.gr.After the beta testing phase has been completed, we willproceed to the full deployment of the platform in its ownspace and roll out an outreach campaign for EAACI mem-bers. Via this process, we will gain additional experienceand feedback that will allow ut to develop the process andthe platform further in the future. Some of the items wehave already identified as future work are listed in the fol-lowing paragraphs.

Argument map visual representation. One part ofthe typical IBIS process we have not yet incorporated is the

visual representation of argument maps. The rationale forthis decision was mainly from a usablity point of view: wewanted to ensure a progressive transition for EAACI usersfrom the current decision making process to an IBIS-basedone. Since most users are already familiar with the kind ofWeb2.0 features we have incorporated in our implementa-tion, we believe the entry barrier will be minimal. After ainitial familiarization period, we will introduce a 2-way in-terpretation process through which dialogue and argumenta-tion graphs developed via the debating process will be visu-alized in a way similar to the typical visulization paradigmsincorporated in other IBIS-based systems. In our case how-ever we will also introduce the additional insight of userfeedback and consensus rate in the visulization. Since thestarting point for this is also the conceptual model of theConsensus ontology, we have already produced the specifi-cations for this 2-way visualization module and are movingforward with the implementation.

Development and integration of a strategic userparticipation and engagement game. We believe thatencouraging and rewarding constructive participation is ofthe utmost important for the operation of any community,as it promotes the sense of belonging and equality and helpsthe common setting of ground rules via social mobility. Inthis respect, we deem the principles of democratic meetingtechniques and normative design (as applied e.g. in [15]) asthe guidelines for the development and application of sucha game model approach. Again, for the case of EAACI weintend to introduce such a strategic game that rewards useractions according to a participation model, after consultingwith EAACI in order to determine the kind of evaluationand reward that is desirable from an organizational pointof view. From a conceptual and technical point of view, wedeem this process to be well-grounded.

Proposal drafting. The Open Innovation deliberationprocess includes support for a proposal drafting, which weconsider an important part of the process as it is meant toconsolidate input received during the initial phases of theprocess in order to compose a text that captures the col-lective consensus on the subject under debate. In order forthis phase to be introduced into the platform, issues of or-ganizational and cultural as well as of technical nature haveto be dealt with. As is usually the case, perhaps the great-est challenge is on the organizational and cultural side, asthe idea of direct involvement in the composition of pro-posals has to grow within organizations. To this end, theEAACI use case is a proof of concept to investigate whetherthe introduction of a non-mediated involvement in the de-cision making process, properly supported methodologicallyand technically, can lead to the development of a partici-patory culture. On the technical side, techniques for textsummarization and composition combined with metrics toincorporate user feedback can be used to this end.

Scaling. What could perhaps be called the final frontierfor collective decision making is scaling up, as this has beenone of the main obstacles for applying such techniques in awide array of contexts, including political decision making.While the use of an ICT backbone can ensure the technicalcapability to accommodate the load this entails, the infras-tructure alone does not suffice to make this a reality with-out an appropriate architecture. In this respect, we deemthe Outcast voting network architecture (see Fig. 8) an im-portant step to that direction and are looking into ways of

integrating with it, as the two approaches are orthogonal.Outcast defines an architecture for a voting infrastructurewith the aim of scaling to communities that are large in num-ber and self-organised in nature. Outcast also supports thenotion of vote delegation and is a sophisticated architecturefor the provision of an integrated environment for Adhoc-racy [20]. However at this stage the architecture focuses onthe voting part, and even though there is an outline for theintegration of drafting and discussion media, this is left un-specified. We believe therefore that this work complementsthe architecture in a very suitable way, not only by provid-ing instances of such a drafting and discussion medium, butalso by semantically specifying the integration points andprocess.

Figure 8: Outcast voting network

Argumentation grounding. Although we have chosennot to pursue an approach geared towards formal argumen-tation, as this would discourage widespread use and adop-tion of our process and platform at this stage, we do notunderestimate its potential. One of our goals therefore isto work on the integration of formal argumentation in theproposed framework, enabling the expression of abstract ar-gumentation by users who are willing and capable to usethis formalism.

Availability. Last but not least, in order for the eDial-ogos Consensus platform to achieve maximum effect and be

used where it is really needed, we are planning to make thecore of the platform available under a dual licensing modethat will ensure free use for non-for-profit causes and organ-isations. The eDialogos Consensus ontology is already avail-able at http://www.imc.com.gr/ontologies/eDialogos/consensusunder a Creative Commons License.

We see this as an ongoing multidimensional process, onethat combines literature and scientific work, technologicalprogress and social reflection in order to provide an evolv-ing aproach to wide-scale collaborative decision-making. Wehope that the approach will be succesful and useful for futurework of our own as well as others in the field.

7. ACKNOWLEDGMENTSWe would like to thank Dr. Nikos Papadopoulos from

EAACI for supporting this work and providing valuable in-sight in the inner working and structure of EAACI and itsdecision making process. We also would like to thank theeDialogos Consensus platform development team, namelyManolis Georgopoulos, Thodoris Germanis, Maria Melabi-oti and Aristotelis Zosakis that made the implementation ofthe platform possible, as well as Marc-Antoine Parent for thediscussion and insights on the Open Innovation deliberationprocess.

8. REFERENCES[1] CROSSROAD Project. Deliverable 2.1: State of the

Art Analysis. http://crossroad.epu.ntua.gr/files/2010/04/CROSSROAD-D1.

2-State-of-the-Art-Analysis-v1.00.pdf, 2011.[Online; accessed 17-2-2011].

[2] Outcast Voting Network. http://zelea.com/project/outcast/\_overview.xht,2011. [Online; accessed 17-2-2011].

[3] G. Anadiotis, P. Alexopoulos, K. Kafentzis,E. Konstantinou, M. Stankovic, and M. Waeckerle.Semantics-powered Virtual Communities and OpenInnovation for a Structured Deliberation Process. InWorkshop on Semantics for Governance and PolicyModelling, ESWC, 2011.

[4] G. Anadiotis, P. Alexopoulos, K. Mpaslis, A. Zosakis,K. Kafentzis, and K. Kotis. Facilitating Dialogue -Using Semantic Web Technology for eParticipation. InExtended Semantic Web Conference (ESWC), 2010.

[5] D. Ayers. IBIS Vocabulary.http://hyperdata.org/xmlns/ibis/, 2006. [Online;accessed 17-2-2011].

[6] V. Belak and V. Svatek. Supporting Self-Organizationin Politics by the Semantic Web Technologies. In 2ndInternational Conference on eParticipation (ePart),2010.

[7] N. Benn and A. Macintosh. Using PolicyCommons tosupport the policy-consultation process: investigatinga new workflow and policy-deliberation data model. InModelling Policy-Making (MPM), 2011.

[8] R. Boero, E. Ferro, M. Osella, Y. Charalabidis, andE. Loukis. Policy Intelligence in the Era of SocialComputing: Towards a Cross-Policy Decision SupportSystem. In Workshop on Semantics for Governanceand Policy Modelling, ESWC, 2011.

[9] H. Chesbrough. The open innovation paradigm. Openinnovation: The new imperative for creating and

profiting from technology.

[10] P. Ciccarese, E. Wu, J. Kinoshita, G. Wong,M. Ocana, A. Ruttenberg, and T. Clark. The SWANBiomedical Discourse Ontology. Journal of BiomedicalInformatics, 41(5):739–751, Oct 2008.

[11] J. Conklin. Dialogue Mapping: Building SharedUnderstanding of Wicked Problems. John Wiley andSons, Ltd., 2005.

[12] J. Conklin, A. M. Selvin, S. B. Shum, and M. Sierhuis.Facilitated hypertext for collective sensemaking: 15years on from gIBIS. In HYPERTEXT 2001,Proceedings of the 12th ACM Conference on Hypertextand Hypermedia, 2001.

[13] A. de Moor, J. Park, and M. Croitoru. ArgumentationMap Generation with Conceptual Graphs: the Casefor ESSENCE. In 4th ICCS Conceptual StructuresTool Interoperability Workshop (CS-TIW), 2009.

[14] T. F. Gordon. The Policy Modeling Tool of theIMPACT Argumentation Toolbox. In ModellingPolicy-Making (MPM), 2011.

[15] K. Hansson, P. Karlstrom, A. Larsson, andH. Verhagen. Actory: A tool for visualizing reputationas a means to formalize informal social behavior. In2nd International Conference on Reputation: Society,Economy, Trust (ICORE, 2011.

[16] N. Karacapilidis, M. Tzagarakis, N. Karousos,G. Gkotsis, V. Kallistros, S. Christodoulou, andC. Mettouris. Tackling cognitively-complexcollaboration with CoPe it! International Journal ofWeb-Based Learning and Teaching Technologies,4:22–38, 2009.

[17] M. Klein. Using Metrics to Enable Large-ScaleDeliberation. In Collective Intelligence InOrganizations: A Workshop of the ACM Group 2010Conference, 2010.

[18] J. Leite and J. Martins. Social abstractargumentation. In 22nd International JointConference on Artificial Intelligence (IJCAI), 2011.

[19] P.-A. Matt and F. Toni. A game-theoretic measure ofargument strength for abstract argumentation. InProceedings of the 11th European conference on Logicsin Artificial Intelligence, 2008.

[20] H. Mintzberg and A. McHugh. Strategy Formation inan Adhocracy. Administrative Science Quarterly,30(2):160–197, 1985.

[21] E. Panopoulou, E. Tambouris, and K. Tarabanis.eParticipation initiatives in Europe: learning frompractitioners. In 2nd International Conference oneParticipation (ePart), 2010.

[22] J. Park. IBIS Common Format.http://projects.kmi.open.ac.uk/hyperdiscourse/

docs/IBIS-0.1.pdf, 2009. [Online; accessed17-2-2011].

[23] I. Rahwan, F. Zablith, and C. Reed. Laying thefoundations for a World Wide Argument Web.Artificial Intelligence, 171:897–921, July 2007.

[24] M. A. Rodriguez, D. J. Steinbock, J. H. Watkins,C. Gershenson, J. Bollen, V. Grey, and B. deGraf.Smartocracy: Social networks for collective decisionmaking. In 40th Hawaii International InternationalConference on Systems Science (HICCS), 2007.

[25] O. Scheuer, F. Loll, N. Pinkwart, and B. M. McLaren.Computer-supported argumentation: A review of thestate of the art. I. J. Computer-SupportedCollaborative Learning, 5(1):43–102, 2010.

[26] S. B. Shum. Cohere: Towards Web2.0 Argumentation.In 2nd International Conference on ComputationalModels of Argument (COMMA), 2008.

[27] R. Siebes. Parademo: e-Democracy Based on aDelegated Expert Selection Process in a Small-WorldNetwork. In e-Democracy’09, 2009.

[28] A. Westerski, C. A. Iglesias, and T. Nagle. The roadfrom community ideas to organisational innovation: Alife cycle survey of idea management systems. ASpecial Issue of the Journal Web-Based Communities.Community-based Innovation: Designing SharedSpaces for Collaborative Creativity., 2011.

[29] A. Wyner, K. Atkinson, and T. Bench-Capon.Semantic Models and Ontologies in ModelilingPolicy-making - A Position Paper. In ModellingPolicy-Making (MPM), 2011.