Kees van Deemter, Dublin, Trinity College, May 2009 What utility can do for NLG: the case of vague...

Kees van Deemter, Dublin, Trinity College, May 2009

What utility can do for NLG: the case of vague language

Kees van Deemter

University of Aberdeen

Scotland, United Kingdom


Two big issues

• Does NLG have proper foundations?– What is its mathematical core?

• Why is language vague?– When does vague language have benefits

over crisp language?


1. NLG and utility

2. Why is language vague?

3. Why should NLG systems be vague?

4. Conclusions


Why bother about “foundations” for NLG?



• Computing is based on– theory of formal languages/computability– Hoare logic/program correctness

• Linguistics is based on– data & stats– the theory of formal languages– mathematical logic/set theory




• Linguistics is based on– data & stats– the theory of formal languages (syntax)– mathematical logic/set theory (semantics)




• Linguistics is based on– data & stats– the theory of formal languages (syntax)– mathematical logic/set theory (semantics)– how about pragmatics?

• Not just what’s true, but what’s appropriate given the circ’s.


• An NLG program “translates” input to linguistic output

• Essentially the problem of choosing the best Form for a given Content. E.g.– McDonald 1987 – Bateman 1997 on Systemic Grammar

• What does this choice depend on?


• What could this choice depend on? – Roughly: which utterance is most “useful”

• In other words, utility

• A perspective seldom pursued. Exceptions:– Kibble 2003 (IWCS-5, referring expressions)– Klabunde 2009 (ENLG-12)

• NLG programs become more expressive, so this perspective now becomes tempting


Example: the hazards of road ice


Example: Roadgritting (Turner et al. 2009, this conference)

Compare1. “Roads above 500m are icy”2. “Roads in the Highlands are icy”

Decision-theoretic perspective:1. 100 false positives, 2 false negatives2. 10 false positives, 10 false negatives

Supposeeach false positive has utility of -0.1each false negative has utility of -2


Example: Roadgritting (Turner et al. 2009, this conference)

Supposefalse positive has utility of -0.1

false negative has utility of -2

Then1: 100 false pos, 2 false neg = -14

2: 10 false pos, 10 false neg = -21

So summary 1 is preferred over summary 2.


Communication in Game Theory (following Crawford & Sobel 1982)

• Set of contents C• Set of forms F• Set of actions A• Speaker strategy S : CF• Hearer strategy H : FA• Two utility functions

US : A[0,1]UH : A[0,1] (not necessarily the same!)


A special case: Vagueness

• Two related questions:

1. Can Game Theory help us explain why language is so often vague?

2. Can Game Theory tell NLG systems when to use vagueness?


• Vagueness: An expression is vague iffit has borderline cases.

– Example of a vague adjective: “poor” is vague because some people are borderline poor.

– Saying this differently: different thesholds for “poor” may be used


Example: Is John poor?

£10.000 p/a

£30.000 p/a

£20.000 p/a

poverty Norm A

poverty Norm B

John

Norm A: “John is poor” is True

Norm B: “John is poor” is False


• Other vague adjectives: ‘large’, ‘small’, ...

• Vague nouns: ‘girl’, ‘giant’, ‘island’,...

• Vague determiners: ‘many’, ‘few’, ...

• Vague adverbs: ‘often’, ‘slowly’, ...

Relevant for any NLG system with “continuous” input– weather forecasts (FOG, Sumtime-Mousam)– patient data (e.g. Babytalk)


From Babytalk corpus

“BREATHING – Today he managed 1½ hours off CPAP in about 0.3 litres nasal prong oxygen, and was put back onto CPAP after a desaturation with bradycardia. However, over the day his oxygen requirements generally have come down from 30% to 25%. Oxygen saturation is very variable. Usually the desaturations are down to the 60s or 70s; some are accompanied by bradycardia and mostly they resolve spontaneously, though a few times his saturation has dipped to the 50s with bradycardia and gentle stimulation was given. He has needed oral suction 3 or 4 times today, oral secretions are thick.” [BT-Nurse scenario 1]


• The question is: “When (if ever) is vague communication more useful than crisp communication?”

• The question is not: “Can vague communication be of some use?”

• Compare Rohit Parikh (2000)

• Ann calls Bob to bring “the blue book”, her only book on topology


Example by R.Parikh (the book scenario)

• Bob only has to search all blue books

• Ann’s instruction reduces the number of books that Bob can expect to have to check.

• Each calls some books blue that the other does not. But they agree on most books.


Ann’s books

blue-Ann=250

blue-Bob=300

blue-Ann-Bob=225

Ann’s books =1000


What’s the utility of “the blue book”?

Compare expected search times1. without this instruction

2. with this instruction

1. Without instruction: ½*1000 = 500

2. With instruction: 9/10*(½*300) + 1/10*(300+(1/2*700)) = 200


In Parikh’s example, “blue” is crisp.

Scenario can be generalised to situations where each allows boundary cases.


Why is language vague?

Barton Lipman (in A.Rubinstein, “Economics and Language”, CUP 2000; working paper “Why is Language Vague” (2006))

When/why does vague communication give higher payoff than crisp language?


One type of answer: conflict

• S and H may have very different utility functions US and UH

• (Crawford & Sobel 1982, Aragones and Neeman 2000): If US and UH are very different, it can be advantageous to hide information– “Our food is healthy!”– “Our burgers are big!”


One type of answer: conflict

• S and H may have very different utility functions US and UH

• Crawford & Sobel 1982, Aragones and Neeman 2000: If US and UH are very different, it can be advantageous to hide information– “Our food is healthy!”– “Our burgers are big!”

• Henceforth (following Lipman): US = UH


Lipman’s questions

When does vague communication give higher payoff than crisp language?

Lipman: the airport scenario


Lipman’s scenario

Example: Airport scenario: I describe Mr X to you, so you can pick up X from the airport. All I know is X’s height; heights are distributed across people uniformly on [0,1]. If you identify X right away, you get payoff 1; if you don’t then you get payoff -1



What description would work best?• Optimal communication: state X’s height

as precisely as possible. If each of us knows X’s exact height then the probability of confusion is close to 0.



What description would work best?

• Optimal communication: state X’s height as precisely as possible. If each of us knows X’s exact height then the probability of confusion at the airport is close to 0.

Lipman: This is not vague, because there are no boundary cases!


Some possible answers to Lipman


1. Necessary vagueness?

“Input may be vague”. E.g.:– verbatim repetition (“hearsay”) – memory may cause details to fade

(e.g., number of casualties in a disaster)– perception may have been inadequate

(e.g., the height of a seated person)


1. Necessary vagueness?

“Input may be vague”. E.g.:– verbatim repetition (“hearsay”)

This begs the question– memory may cause details to faded

(e.g., number of casualties in a disaster) ?– perception may have been inadequate

(e.g., the height of a seated person)Lipman: Why can’t we convey exactly what our perception/memory is?

E.g. “24 degrees +/- 2 degrees”


the hazards of measurement 11m 12m


• Example: One house of 11m height and one house of 12m height

1. “the 12m house needs to be demolished”

2. “the tall house needs to be demolished”

• Comparison is easier and more reliable than measurement prefer utterance 2 (Van Deemter 2006)


• Example: One house of 11m height and one house of 12m height

1. “the 12m house needs to be demolished”

2. “the tall house needs to be demolished”

• Comparison is easier and more reliable than measurement prefer utterance 2

• But arguably, this utterance is not vague

Its vagueness is merely local


Apparent vagueness is frequent

• ‘the tall house’ the tallest house

• ‘Physical exercise is good for young and old’ regardless of age

• ‘Bad for bacteria, good for gums’ gums improve as a result of bacterial death

• ‘Fast-flowing rivers are deep’ the faster the deeper (positive correlation between variables)


3. Production/interpretation Effort

• GTh can reason about the utility of an utterance• Effort needs to be commensurate with utility. In

many cases, more precision adds little benefit (cf. Prashant Parikh 2000, Van Rooij 2003, Jaeger 2008)

• E.g., the feasibility of an outing does not depend on whether it’s 20C or 30C.

• ‘Mild’ takes fewer syllables than ‘twenty three point seven five’.– Vague words tend to be short (Krifka 2002)


But: Why not round the figure?

“The temperature is 24C”


4.Evaluation payoff

• Example: You ask the doctor about your blood pressure.– Utterance 1: “Your blood pressure is 150/90.”– Utterance 2: “Your blood pressure is high.”

• U2 offers less detail than U1• But U2 also offers more: an evaluation of

your condition.– A link with actions (cut down on salt, etc.)– Especially useful if metric is “difficult”


– But why does English not have a (brief) expression that says “Your blood pressure is 150/90 and too high”?

Compare“You are obese”

means

“Your BMI is above 30 and this is dangerous”.


5.Lack of a good metric

• Maybe areas where there exists a generally accepted measurement are rare– Multidimensional measurements: What’s the

size of a house?– Maths: How difficult is a proof? (“As the

reader may easily verify”)


5.Lack of a good metric

• Maybe areas where there exists a generally accepted measurement are rare– Multidimensional measurements: What’s the

size of a house?– Maths: How difficult is a proof? (“As the

reader may easily verify”)– How beautiful is a sunset?


6.Future contingencies

• Indecent Displays Control Act (1981) forbids public display of indecent matter


• Indecent Displays Control Act (1981) forbids public display of indecent matter

• Indecent at the time

the law has been parameterised

(Waismann 1968, Hart 1994, Lipman 2006)


7. Vagueness facilitates search


7. Vagueness facilitates search

Palace scenario: A diamond has been stolen from the Emperor. The thief must have been one of the 1000 eunuchs. The sole witness gets wounded and says, with his last breath, “the thief is tall”.

What should the emperor do?


What if ‘tall’ is crisp?

• Separate eunuchs: tall / not tall

tall 500

not tall 500

• Expect to search ½*500=250 tall eunuchs

• What if witness regards more people as tall?

• Seach 500 tall ones + 250 not-tall ones

• No difference between the “non-tall” ones


What if ‘tall’ is recognised as having borderline cases?

• Separate: tall / not tall / borderline

1. tall 100

2. borderline-tall 400

3. not-tall 500

Assume: probability of being called tall is highest for (1) and lowest for (3).


What if ‘tall’ is recognised as having borderline cases?

• Separate: tall / not tall / borderline

1. tall 100

2. borderline-tall 400

3. not-tall 500

Assume: probability of being called tall is highest for (1) and lowest for (3).

E.g., assuming the probabilities in the figure: expect to search 0.5*50+0.5(100+200)=175

p = 0.5

p = 0.5

p = 0


What if ‘tall’ has a continuity of degrees?

• This would allow the emperor to rank eunuchs by their height:

(1) tallest

(2) tallest but 1

(3) tallest but 2

...

(n) shortest

Assume: probability of being called tall is highest for (1), second-highest for (2), etc.

Emperor should search (1) first, then (2), etc.


First lesson

• Continuous domains invite differences between subjects, regarding their interpretation thresholds

• Stubborn insistence on your own thresholds is foolish: flexibility pays

• Minimal flexibility: tall/not tall/borderline

• Maximal flexibility: continuum of degrees


Second lesson

• The emperor’s three search strategies correspond with three logics of vagueness:

• Knowledge as ignorance: “tall” is really crisp; we just don’t know where the threshold is.

• Partial Logic: true, false, truth value gap.

• Degree theories (e.g. Fuzzy Logic).


Second lesson

• The emperor’s three search strategies correspond with three logics of vagueness:

• Knowledge as ignorance: “tall” is really crisp; we just don’t know where the threshold is. Corresponds with 1st strategy

• Partial Logic: true, false, truth value gap. Corresponds with the 2nd strategy

• Degree theories (e.g. Fuzzy Logic). Corresponds with 3rd strategy


Second lesson

• This suggests that Knowledge as Ignorance is probably “wrong”, and so is Partial Logic.


Relevance for NLG

• We have focussed on the question “Why is language vague?”

• What does this mean for NLG?– Do our answers translate directly to the

question when NLG systems should use vagueness?


Vagueness triggers for NLG?

1. Necessary vagueness

2. Apparent vagueness

3. Cost reduction

4. Bias/evaluation

5. Lack of good metric

6. Future contingencies

7. Facilitation of search



1. Necessary vagueness. Yes: When NLG systems take vague information as starting point. (e.g.: Colour(x)=red)

2. Apparent vagueness. Yes: vague descriptions (e.g. “the figure on the left”)

3. Cost reduction: Yes: the fact that English could have had a short expression for “24 degrees +/- 2” is irrelevant.



4. Bias/evaluation. (Similar to 3.) the fact that English could have had an expression for “these shoes cost 200 pounds and I disapprove of that” is irrelevant. The generator can only choose between existing expressions: ”200 pounds”, and “expensive”



5. Lack of good metric. Relevant to NLG in principle, given challenging applications. (E.g. estate adverts)

6. Future contingencies. Similar to (5).7. Facilitation of search. Relevant(?)

unless system defines its terms explicitly, removing differencs between subjects (e.g., Reiter et al. 2005, the word “evening”)


Towards an algorithm

• In what follows, just a few factors have been taken into account

• Caveat: Any resemblance with empirically supported facts is accidental


Towards an algorithm

uncertainty precision evaluation cost total penalty payoff benefit

“39.82C” -1 9 0 -4 4

“approx. 40C” 0 8 0 -3 7

“high fever” 0 6 2 -2 8


Written material

“What Game Theory can do for NLG: the case of vague language”. Proc. 12th European workshop on Natural Language Generation (ENLG-2009)


Not Exactly:In Praise of Vagueness

Oxford University Press To appear Jan 2010

Part 1: vagueness in science and daily life

Part 2: logic and vaguenessPart 3: vagueness and AI


Not Exactly:In Praise of Vagueness

Oxford University Press To appear Jan 2010

Chapter 11: “When to be vague?”


• “Payoff” could become central to NLG• New research by game theorists,

economists, logicians, linguists (e.g., Lipman, Van Rooij, Jäger, de Jaegher, Aragones, Neeman)


• Lipman: “Why is language vague?”

• Tentative answers are beginning to emerge

• Do not confuse with the question “When should an NLG system (or a human speaker) be vague?”

since this depends on what the language can express


• Lipman: “Why is language vague?”• Tentative answers are beginning to

emerge• Do not confuse with the question

“When should an NLG system (or a human speaker) be vague?”

since this depends on what the language can express

• Compare De Saussure: langue/parole


Thanks ...

• to Ehud Reiter and Albert Gatt.

Kees van Deemter, Dublin, Trinity College, May 2009 What utility can do for NLG: the case of vague...

Documents

Transcript of Kees van Deemter, Dublin, Trinity College, May 2009 What utility can do for NLG: the case of vague...