Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California...
-
Upload
jeremy-wells -
Category
Documents
-
view
223 -
download
0
Transcript of Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California...
![Page 1: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/1.jpg)
Issues in Dialogue Management Issues in Dialogue Management for Multiparty Dialoguefor Multiparty Dialogue
David TraumUniversity of Southern
CaliforniaInstitute for Creative Technologies (ICT)[email protected]
![Page 2: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/2.jpg)
ICT Mission Rehearsal Exercise (MRE) Project
•VR Theatre•8’ 150˚ Curved Screen, •Multiple Projectors•10-2 3-d spatialized sound
Bosnia Scenario (Swartout et al ‘01)Human lieutenant (student) faces peacekeeping dilemmasArtificial agents interact with user
Mentor (e.g., sergeant, front left)Teammates (e.g., medic, front right)Locals (e.g., mother, front center)
![Page 3: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/3.jpg)
Aspects of MRE DialogueAspects of MRE Dialogue
Multimodal: Face To Face (speech+gesture), Radio Speaking modes (shouting, normal, whispering)
Interleaved communication and action Communication to support action (orders, negotiation) Actions to support communication (contact, turn-taking) Actions as communication (acting on an order as
grounding order) Multiple Interactors
Messages tailored for multiple addressees/overhearers Multiple Conversations
LT With base/other platoon about arrival time, medevac
LT With Sgt, Medic about local area/platoon orders
SGT with troops to carry out orders
![Page 4: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/4.jpg)
Dialogue issuesDialogue issues
Participation Roles Speaker ID Addressee ID Participant status
Multiple conversations & threadsChannel management Turn-takingInitiativeObligationsGroundingAttention
![Page 5: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/5.jpg)
Participant Roles Participant Roles (Goffman 74, 81, (Goffman 74, 81, Clark 96)Clark 96)
Speaker & Hearer are really complex composites Not individual roles Different kinds of participant status
Different rights and responsibilities & actions
![Page 6: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/6.jpg)
Speaker sub-rolesSpeaker sub-roles
Roles Composer Performer Responsible Agent Ratified/unratified
Examples of split roles Author/performer Speechwriter/politician Foreign language speaker/interpreter Copywriter/spokesman/owner
![Page 7: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/7.jpg)
Hearer sub-rolesHearer sub-rolesRoles
Addressee (spoken directly to) Side participant (ratified) Bystander (tolerated) Eavesdropper (unknown)
Issues: who gets/has/does/is Signals from speaker Obligations to speaker Right to become speaker Speaker intend to hear (or intends not to hear) Message designed for Speaker awareness Attention of participants
![Page 8: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/8.jpg)
Speaker -> Addressee signalsSpeaker -> Addressee signals
Vocatives & semantic indicationsMessage tailored for understandingBody orientationGazeGestureMirroring
![Page 9: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/9.jpg)
Addressee -> Speaker signalsAddressee -> Speaker signals
Attention (ratification) Gaze Posture/orientation mirroring
Uptake Nods, head shakes Facial expressions Eyebrow flashes
Turn-taking Feedback Hands in gesture space gaze
![Page 10: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/10.jpg)
Speaker IDSpeaker ID
Two Party: If not me, then you
Multi-party: Audio
Acoustic features Self ID Style features/content
Multi-modal Stereo localization Visual identification (lips moving, gesturing)
![Page 11: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/11.jpg)
Addressee IdentificationAddressee Identification
Two-party: Non-speaker
Multi-party Speech/Text
Vocatives Content Context
Multimodal Gaze Orientation gesture
![Page 12: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/12.jpg)
MRE Multi-party (speech or text) MRE Multi-party (speech or text) Addressee Identification: Addressee Identification: AlgorithmAlgorithm
1. If utterance specifies addressee• Vocative• not expecting short answer or clarification
of person type Addressee = specified addressee
2. Else If current utterance speaker is same as previous utterance speaker Addressee = previous addressee
3. Else If previous speaker current speaker Addressee = previous speaker
4. Else if (active) conversational participant in same conversation Addressee = participant
5. Else ?
![Page 13: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/13.jpg)
Participant RolesParticipant Roles
Conversational Rolestask roles
authority, responsibility, participant, desire, guard
social roles Status: superior, subordinate, equal, incomparable
Closeness: friend, comrade, colleague, acquaintence,stranger, opponent, antagonist
activity roles e.g. courtroom: judge, bailiff, lawyer, witness
![Page 14: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/14.jpg)
Change in Participant StatusChange in Participant Status
Turn-taking Addressee -> speaker
Speaker selection Self-selection
Other -> speaker Speaker -> addressee (or other)
Addressee -> other Speaker addressee shift Addressee attention shift
Other -> addressee Addressee-like behavior
Attention, grounding Speaker inclusion
![Page 15: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/15.jpg)
Activity-oriented talkActivity-oriented talk
Main Activity -ratified speakers & addressees Offline (among speakers, not meant for ratified listeners)
Byplay - ratified addresses & side participants Borderplay (Brandt) - addressees & other ratified
Sideplay - unratified overhearersCrossplay - ratified & unratified
![Page 16: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/16.jpg)
Example Vhuman Inclusion Example Vhuman Inclusion behaviorbehavior
![Page 17: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/17.jpg)
Multi-threadingMulti-threading
Two-Party Topic-shifts
Multi-party Multiple active threads Separate conversations Parallel conversations
Dependency influence
Dynamic starting, ending, splitting, merging, entry, exit
![Page 18: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/18.jpg)
Conversation/thread Conversation/thread IdentificationIdentification
Two-party Single conversation Topic coherence
Multi-party Channel/conversation relationship Addressee/conversation relationship Topic/conversation relationship
![Page 19: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/19.jpg)
Channel managementChannel management
Two-Party Who speaks when
Multi-party Which conversation is active
Modality Cross modality channels Same modality channels
![Page 20: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/20.jpg)
Ft Rucker Helicopter Mission Ft Rucker Helicopter Mission SimulationSimulation
![Page 21: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/21.jpg)
Turn-takingTurn-taking
Model: Cues (basic physical performances) Signals (cluster of one or more cues indicating intent regarding turn)
acts (turn-taking results, given context of all participants actions)
Two-party Take-turn, hold-turn, release-turn
Multi-party Assign-turn, request-turn Management across channels Management across conversations
![Page 22: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/22.jpg)
InitiativeInitiative
Two-party System, user, mixed
Multi-party Asymmetric Cross-initiative
Address different participant Different participant interjects
Cross-conversation initiative
![Page 23: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/23.jpg)
Addressee ObligationsAddressee Obligations
Two-party Addressee has obligation to act
Multi-party Obligations from multi-addressee?
Indefinite obligation (group obligation)?
Distributed obligation to all? No obligation (option)?
![Page 24: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/24.jpg)
GroundingGrounding
Two-party existing models, e.g. Traum&Allen 92 Signals of understanding from addressee needed for grounding
Multi-party signals from whom? One participant? All?
![Page 25: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/25.jpg)
Multi-party grounding modelMulti-party grounding model
Implemented: Multiparty conversation, single addressee
Components: State Initiator Responder Contents
Multi-addressee Any addressee acknowledgement grounds Split into multiple single speaker-addressee units
Cross-grounding
![Page 26: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/26.jpg)
EvaluationEvaluation
Two party Task success Naturalness Efficiency Usability
Multi-party Individual or combined measures? Total effort or real-time?
![Page 27: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/27.jpg)
InteractionsInteractions
Attention + multi-threading, addressee ID
Multi-threading + groundingStatus + turn-takingChannel + addressee + conversation ID
![Page 28: Issues in Dialogue Management for Multiparty Dialogue David Traum University of Southern California Institute for Creative Technologies (ICT) traum@ict.usc.edu.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649e7a5503460f94b7b32c/html5/thumbnails/28.jpg)
SummarySummary
Multiparty dialogue adds new issues Added complexities to two-party issues
New problemsMultiparty also offers new opportunities Insight to interplay of functions, some of which might be latent in two-party interaction
More work needed!