1. An Online DeliberationFacilitatorsDashboard: Visualizations andText analysis to support qualitydialoguesWing, L., Murray, T., Woolf, B., Katsh, E.The Twelfth International Online Dispute Resolution Forum,Montreal, June 2013

2. The Fourth Party: Improving Computer-MediatedDeliberation through Cognitive, Social andEmotional Support 3-Year NSF Social Computing grant, startedFall 2010 Description atwww.socialdeliberativeskills.com2

3. Project collaborators Beverly Woolf: CompSci, PI (intelligent and collaborativeeducational systems) Tom Murray: CompSci; project manager/co-PI, principalvisionary and instigator (ed-tech, cog-psych & D&D) Ethan Katsh (ODR), Legal Studies, co-PI Leah Wing (social justice and ODR), Legal Studies, co-PI Linda Tropp, Psychology of Peace and Violence, advisor(intergroup relations/conflict) Zan Goncalves, New England Center for Civic Life(teaching, practice and study of deliberative democracy) Idealogue Inc.; iCohere. Inc. softwareplatforms(Advanced dialogue)3

4. Thanks for consultationand/or data from: DemarsAssociates.com/PayPal/ebay (e-commerce) Juripax.com (online workplace and divorcesettlements) National Mediation Board (transportationmanagement/labor disputes) Modria.com (e-commerce plus) Idealogue.com (depth-oriented online dialogueplatform) iCohere.com (online communities and work groups) Mass Dept. of Dispute Resolution (civic engagement) New England Center for Civic Life (teaching, practiceand study of deliberative democracy)4

5. Social Deliberative Skills:Social/Emotional/ReflectivePerspective taking & cognitive empathyPerspective seeking (curiosity/inquiry)Self-reflection: on ones biases, intentions,emotional stateMeta-dialog: Reflect on the quality of thedialogEpistemic skill: e.g. treating facts/datadifferently from opinions/hypothesesTolerance for uncertainty, ambiguity,disagreement, paradox5

6. Support/Scaffolding vs.EducationFacilitatedOnlineDELIBERATIONOutcomes:- Agreements/solutions- Relationship, Trust (social capital),understanding- SKILL USE (and practice)ExistingSkillsAdaptiveSupport(4th party)InterfaceSupportFacilitatorSupport(Dashboard)

7. Example: Topics Chosen by Studentsset context in Spring 12 Week 1: Discuss the pros and cons of legalizingmarijuana Week 2: Sex whats the big deal? What values aremost important in making sexual choices? Week 3: Discus the pros and cons of the death penalty(capital punishment)7

8. [CURRENT] WEEK 1: Discuss the pros and cons of leg...UPDATE PROFILELOG OUTHOMELogged in as tomm[CURRENT] WEEK 1: Discuss the pros and cons of legalizing marijuana.[CURRENT] WEEK 1: Discuss the pros and cons of legalizing marijuana.To focus the conversation, we invite you to assume you are on an advisory panel for the statelegislature, having some preliminary conversations online, and you will eventually be draftinga group recommendation. Consider not only your own preferences but what is best for thestate (or society).edit deleteCONTRIBUTE YOUR THOUGHTS14:53 EDT Sunday, November 13 by tommtomm has joined the conversation23:53 EDT Saturday, November 12 by ines- vines-v added a resource: Getting a Fix23:52 EDT Saturday, November 12 by ines- vI have to disagree with your third point that marijuana is a gateway drug. Ofall the people I know that smoke marijuana, they do not do any hard drugs.I do agree that gateway drugs exist, however I feel like that typicallyhappens from one hard drug to another when one doesnt seem to beenough. But if you want to talk about gateway drugs we would also have tomention alcohol and cigarettes which many people consume and smoke.Alcohol and cigarettes are also drugs and often considered gateway drugs.They are both legal so that option is void in regards to marijuana.You also mentioned cancer and other lung related issues. Marijuana is anatural plant. Cigarettes are made up of extremely harmful chemicals thatcause lung related issues and cancer much faster than marijuana ever could.Yet, they are still legal. If anything, cigarettes should be illegal whenconsidering public health. Marijuana is a lot safer than cigarettes.I do appreciate you playing Devils advocate though!Id like to explain how I see it differently (ines-v)18:26 EDT Friday, November 11 by arthur- xIt seems like the vast majority is supportive of the legalization of marijuana,so Im going to play devils advocate in order to bring the oppositions sideto the table.First off, research has demonstrated that marijuana use reduces learningability by limiting the capacity to absorb and retain information. A 1995study of college students discovered that the inability of heavy marijuanausers to focus, sustain attention, and organize data persists for as long as 24hours after their last use of the drug. Earlier research, comparing cognitiveabilities of adult marijuana users with non-using adults, found that users fallshort on memory as well as math and verbal skills. Although it has yet to beproven conclusively that heavy marijuana use can cause irreversible loss ofintellectual capacity, animal studies have shown marijuana-inducedines-varthur-xjoseph-tlaura-trtwellsmatthew-stommDIALOGUE TABLEEveryone (no demographics set)8

9. 9MediemOpinion Sliders

10. Experimental ConditionsExp Group N Gender GradeVanilla 8 (5 Female,3 Male)4 soph, 4 juniors,0 seniorsReflective Tools 8 (5 Female,3 Male)4 soph, 2 juniors,2 seniors(Sliders) 8 (Group omitted due to interaction issues)10 V&R groups: 241 posts and 516 segments (averageof 15.06 (SD = 7.45) posts/student) Mean words/post = 54 (SD = 42); meancharacters/post = 299 (SD = 242)

11. Text Coding11Soc. DELIBERATIONSkill EvidenceMISC CODES ACTIONNEGOTIATIONARGUMENT CODESSELF_REFLection_INTERSUBictiveQ_INTERLocutorREF_INTERLocutorPERSPECTIVE_taking_META_DialogMEDIATEMETA_CONSMETA_CONFLMETA_SUMMETA_CHECK_META_TOPICWEIGHSYSTEMs_thinkingFACT_cite_SouRCeSOURCE_REFerenceAPPRECiationQ_TOPICCHANGE_mindUNCERtaintyOTHERS_THNKAPOLOGYHELPREQ_HELPPROCESSAGREEDISAGREE_NEGative-emotionNEGEMO_INTerlocutorNEGEMO_Topic_OFFTOPICTECHnicalSOCIAL(External actions)ActRequestActProposeActAcceptActDeclineActNegot(Dialogue_Actions)DI_ActRequestDI_ActProposeDI_ActAcceptDI_ActDeclineDI_ActNegot(Facilitators only)WELCOMINGPROC_EXPLMOTIVATE_ARGument_GENericFACT_NOSRCGENERAL_SOLUTNEXPER_OBSERVARG_OPINIONOPINION_ONLYOVER_GENSUPPORTSUM_MY-argumtEXAMPLEELAB

12. Main EffectExp. Group Total_SD_SkillIntersubjectivespeech actsVanilla (N = 8) 0.29 (0.07) 0.20 (0.09)Reflective Tools (N = 8) 0.40 (0.08) 0.30 (0.08)12 A significant difference and main effect betweenTotal-SD-Score and grouping, F(1, 14) = 6.89, p =0.02*, d = 1.46 (a large effect) in favor of theReflective Tools group A significant relationship between Intersub andgrouping, F(1, 14) = 4.81, p = 0.05*, d = 1.05 (a largeeffect) in favor of the Reflective Tools group

13. 13Facilitators Dashboard

14. Dashboard Text Tagging14

15. Advice Screen

16. Applicability and UseFacilitators To identify individual and group participationlevels for assessing useful interventions(ie: to stimulate more involvement, moreconstructive engagement, etc.) To identify patterns of silence or intensity(ie: to ID topics/relationships to attend to;use as clues to search text analysis forconflicts or breakthroughs)

17. Applicability and UseParticipants For self/group assessment of engagement,topical, and relationship patterns re:participation, intensity, silence For clues to search text analysis forconflicts or breakthroughs

18. CohMetrixdiscourse & coherence18LIWClexical categories

19. Skill vs. text metrics correlations

20. LIWC/Cohmetrix Correlations withTotal-SD-Skill

21. Cross-DomainTrainingPrelim. Machine Learning ResultsFor Predicting Total Skill

22. Future ApplicationsCommon problems encountered in online facilitation Low or no participation of individuals or groups, orsilences or lulls on the part of individuals, the entiregroup, or sub-groups Conversation domination by an individual or group Inappropriate or disrespectful behavior Off-topic conversation Tension-filled disagreements, or high emotionalcontent Too much agreement or politeness Misunderstanding due to missing communication skillsnormally available in face-to-face communication Violation of rules (e.g. confidentiality, no advertising,etc.)

23. Thank you

24. Extra slides

25. Automated Text AnalysisLIWC (Pennebaker et al.) Dictionary-based (linguisticinquiry word count) 4,500 words/STEMS; 80 word categories we focus on 19 of them 80 >> 4 general descriptor categories (word count, words per sentence, % of words captured, and %of words >6 letters), 22 standard linguistic dimensions (e.g., % pronouns, articles, auxiliaryverbs, etc.), 32 psychological constructs (e.g., affect, cognition, biological processes), 7 personalconcern categories (e.g., work, home, leisure activities), 3 paralinguistic dimensions(assents, fillers, nonfluencies), and 12 punctuation categories (periods, commas, etc). Relate the categories to things like aggression, used in theraputic contexts, to ID lying, Coh-Metrix (Graesser et al.) syntax, referential cohesion, semantic cohesion, rhetorical composition 100 measurements categories We focus on 4 composite measurements (or majorfactors): Narrativity, Referential Cohesion, SyntacticSimplicity, and Word Concreteness

26. Experimental Groups27

27. Code FrequenciesIntersub Meta_DialogueMeta_TopicApology AppreciationFact_SourceSource_Ref#students22 5 15 1 8 1 4% ofsegs 25% 0.9% 5.5% 0.2% 1.3% 0.3% 1.2%28

28. Total Skill score adds: Appreciation (Gratitude, affirmation of anothers ideaor situation) Apology Fact--sourced (stating a fact and noting the source inthe same post) Source Reference (Mentioning a source, with areference or description; without a fact) Intersubjectivity: perspective taking or question asking Meta-dialogue, discussing the quality of the dialogue Meta-Topic: Birds eye or systemic view of the topic29

29. Next: Linked Representations clicking on the name of an individual or group in a chart ornetwork diagram will focus (or filer or highlight) all tools onthat individual or group; clicking on a link in the network diagram will show postsbetween the relevant interlocutors; clicking on a word in the word cloud will highlight postsincluding that word; clicking on a location in the time-axis of a trend line willnavigate to posts in the Timeline at that time; hovering over an agent trigger will show the Advice or Alertassociated with that event; and clicking on Advice or Alertswill navigate to the place in the dialogue timeline for thetriggering event(s).

30. Text Coding31Soc. DELIBERATIONSkill EvidenceMISC CODES ACTIONNEGOTIATIONARGUMENT CODESSELF_REFLection_INTERSUBictiveQ_INTERLocutorREF_INTERLocutorPERSPECTIVE_taking_META_DialogMEDIATEMETA_CONSMETA_CONFLMETA_SUMMETA_CHECK_META_TOPICWEIGHSYSTEMs_thinkingFACT_cite_SouRCeSOURCE_REFerenceAPPRECiationQ_TOPICCHANGE_mindUNCERtaintyOTHERS_THNKAPOLOGYHELPREQ_HELPPROCESSAGREEDISAGREE_NEGative-emotionNEGEMO_INTerlocutorNEGEMO_Topic_OFFTOPICTECHnicalSOCIAL(External actions)ActRequestActProposeActAcceptActDeclineActNegot(Dialogue_Actions)DI_ActRequestDI_ActProposeDI_ActAcceptDI_ActDeclineDI_ActNegot(Facilitators only)WELCOMINGPROC_EXPLMOTIVATE_ARGument_GENericFACT_NOSRCGENERAL_SOLUTNEXPER_OBSERVARG_OPINIONOPINION_ONLYOVER_GENSUPPORTSUM_MY-argumtEXAMPLEELAB

31. Social Deliberative Skills:Social/Emotional/Reflective 1. Social perspective taking(cognitive empathy, reciprocal roletaking...) 2. Social perspective seeking (socialinquiry, question asking skills...) 3. Social perspective monitoring(self-reflection, meta-dialogue...) 4. Social perspective weighing(reflective reasoning; comparing andcontrasting views...)32

32. Domain statistics andinter-rater agreement

33. Codoolecoding tools34Coole coding tools

34. Social Deliberative Skill:application of HOSs to me/you/weHigher Order Skills argumentation critical thinking explanation & clarification inquiry/curiosity(question asking & investigation) reflective judgment meta-cognition epistemic reasoningApply these skills, not toEXTERNAL REALITY (IT/problemdomain) but to theINTERSUBJECTIVE domainHigher Order Skills applied to:SELFgoals; level of certainty;feelings, values, assumptionsYOUgoals, assumptions, feelings,values; perspective taking;"believing" & cognitive empathyWEagreements, goals; quality ofthe discourse/collaboration;differences and similarities invalues, beliefs, goals, power, roles

35. Survey Questions29 of the 36 participants took the survey

36. By Demographic

37. Settings

38. Post number vs. size