1

Natural Language Processing

Dialog

2

Today

• Spoken dialogue systems

3

Talking to Computers

• Spoken dialogue systems make it possible to accomplish real tasks without talking to a real person

• Keys to success– goal-directed interactions in a limited

domain– Priming users to adopt a vocabulary

you can recognize– Segmenting the task into manageable

stages– Judicious use of system vs. mixed

initiative

4

Dialogue vs. Monologue• Monologue and dialogue both involve

interpreting– Information status (given and new info)– Coherence issues– Reference resolution– Speech acts, implicature, intentionality

• Dialogue involves managing– Turn-taking– Grounding – Detecting and repairing misunderstandings– Initiative and confirmation strategies

5

Example

• Here’s an (unfair) example from the ATT Toot system.

6

Segmenting Speech into Utterances

• What is an `utterance’?– Single syntactic sentence may span

several turnsA: We've got you on USAir flight 99B: YepA: leaving on December 1.

– Multiple syntactic sentences may occur in single turn

A: We've got you on USAir flight 99 leaving on December. Do you need a rental car?

– Utterance segmentation: cue words, n-gram word or POS sequences, prosody (pitch, accent, phrase-final lengthening, pause duration)

7

Turns and Utterances

• Dialogue is characterized by turn-taking: – Who should talk next– When they should talk

• Turns in recorded speech:– Little speaker overlap (around 5% in

English)– But little silence between turns either

8

Turn Taking

• How do we know when a speaker is – Giving up or taking a turn? – Holding the floor? – Interruptable?

• How do I know when– Its my turn obligatorily– Optionally?

9

Simple Turn-Taking Rules• At each transition-relevance place

(TRP) of each turn:– If current speaker has selected A as next

speaker, then A must speak next– If current speaker does not select next

speaker, any other speaker may take next turn

– If no one else takes next turn, the current speaker may take next turn

• TRPs are where the structure of the language allows speaker shifts to occur

10

Turn Taking Scripts• Adjacency pairs set up next speaker

expectations– GREETING/GREETING– QUESTION/ANSWER– COMPLIMENT/DOWNPLAYER– REQUEST/GRANT

• Significant silence is dispreferredA: Is there something bothering you or not?

(1.0s)A: Yes or no? (1.5s)A: Eh?B: No.

11

Overall System Strategies

• System Initiative (Control freak)S: Please give me your arrival city name.U: Baltimore.S: Please give me your departure city nameU: BostonS:…

• Rigid, unnatural, difficulty with chatty users

12

Overall System Strategies

• Mixed initiativeS: How may I help you?U: I want to go to Boston.S: What day do you want to go to Boston?

• User InitiativeS: How may I help you?U: I want to go from Boston to Baltimore on

November 8.

13

Grounding

• Participants are trying to come to a meeting of minds, they’re trying to establish common ground (or a set of mutual beliefs)

14

Grounding

• Hearers must ground a speaker’s utterances by making it clear whether or not understanding has occurred

• Various ways to do this…

S: I can upgrade you to an SUV at that rate.

User: ????

15

Grounding (Clark and Schaefer 1989)

S: I can upgrade you to an SUV at that rate.– Continued attention/permission to proceed

(User gazes appreciatively at S)– Relevant next contribution

U: Do you have an Explorer available?– Acknowledgement/backchannel

U: Ok/uh-huh/Great!– Display/repetition

U: An SUV? U: You can upgrade me to an SUV at the same rate?

– Request for repairU: Huh?

16

Evaluation

• Performance of a dialogue system is affected both by what gets accomplished by the user and the dialogue agent and how it gets accomplished

MaximizeMaximizeTask SuccessTask Success

Minimize Minimize CostsCosts

EfficiencyEfficiencyMeasuresMeasures

QualitativeQualitativeMeasuresMeasures

17

Metrics

• Efficiency of the Interaction:User Turns, System Turns, Elapsed Time

• Quality of the Interaction: ASR rejections, Time Out Prompts, Help Requests, Barge-Ins, Cancellation Requests, …

• User Satisfaction• Task Success: perceived completion,

information extracted, information learned (in tutorial setting)

18

User Satisfaction Metrics• TTS Performance

– Was system easy to understand in this conversation?• ASR Performance

– In this conversation, did system understand what you said?• Task Ease

– In this conversation, was it easy to do what you wanted?• Interaction Pace

– Was the pace of interaction appropriate in this conversation?

• User Orientation– In this conversation, did you know what you could say at

each point of the dialog? • System Response

– How often was the system sluggish and slow to reply to you in this conversation?

• Expected Behavior– Did system work the way you expected it to in this

conversation?

19

Misrecognition Repair

• Recognizing when the conversation has gone astray and recovering…

– Mainly by analyzing the user’s utterances for signals that things are going astray

20

Wrapup

• Spoken dialogue systems present new problems -- but also new possibilities– Recognizing speech introduces a new

source of errors– Additional information provided in the

speech stream offers new information about users’ intended meanings, emotional state (grounding of information, speech acts, reaction to system errors)