Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics...
Transcript of Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics...
![Page 1: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/1.jpg)
Compositional Lexical Semantics for Natural Language Inference
Thesis Defense Ellie Pavlick
Department of Computer and Information Science University of Pennsylvania
![Page 2: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/2.jpg)
what is the population of new york city?
![Page 3: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/3.jpg)
what is the population of new york city?
![Page 4: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/4.jpg)
what is the population of new york city?
![Page 5: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/5.jpg)
what is the population of new york city?
“What is Saturday afternoon’s forecast?”
“Will it be sunny this weekend in Miami?”
“What’s the weather going to be like this
weekend?”
how many people live in nyc?
new york city population
how big is new york city? how crowded is ny?
number of residents of nyc
“Is it going to be nice out on Saturday?”
Human language is highly variable.
![Page 6: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/6.jpg)
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 7: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/7.jpg)
in hacked fundraiser recording • in leaked
recording • in audio from hacked email • privately •
hacked audio:
mocks • said • insults • characterizes • comments on •
gives frank take on • slams • calls • knocks • describes
bernie supporters • millennials •
sanders supporters • young voters • bernie sanders
supporters • bernie kids • bernie fans
losers who live in their parents'
basements • basement dwellers • frustrated basement-
dwellers • basement-dwellers & baristas
Hillary • Hillary Clinton • HRC
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 8: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/8.jpg)
in hacked fundraiser recording • in leaked
recording • in audio from hacked email • privately •
hacked audio:
mocks • said • insults • characterizes • comments on •
gives frank take on • slams • calls • knocks • describes
bernie supporters • millennials •
sanders supporters • young voters • bernie sanders
supporters • bernie kids • bernie fans
losers who live in their parents'
basements • basement dwellers • frustrated basement-
dwellers • basement-dwellers & baristas
Hillary • Hillary Clinton • HRC
In leaked audio, Clinton talks about Sanders supporters “living in basement”
How to we know when two different expressions in natural language have
the same meaning?
![Page 9: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/9.jpg)
in hacked fundraiser recording • in leaked
recording • in audio from hacked email • privately •
hacked audio:
mocks • said • insults • characterizes • comments on •
gives frank take on • slams • calls • knocks • describes
bernie supporters • millennials •
sanders supporters • young voters • bernie sanders
supporters • bernie kids • bernie fans
losers who live in their parents'
basements • basement dwellers • frustrated basement-
dwellers • basement-dwellers & baristas
Hillary • Hillary Clinton • HRC
In leaked audio, Clinton talks about Sanders supporters “living in basement”
How to we know when two similar expressions in natural language have
a different meaning?
![Page 10: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/10.jpg)
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 11: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/11.jpg)
In leaked recording=
Logical Inference
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 12: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/12.jpg)
In hacked fundraiser recording
In leaked recording⊂
=
Logical Inference
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 13: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/13.jpg)
PrivatelyIn hacked fundraiser recording
In leaked recording⊂ ⊂
=
Logical Inference
In leaked audio, Clinton talks about Sanders supporters “living in basement”
![Page 14: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/14.jpg)
PrivatelyIn hacked fundraiser recording
In leaked recording⊂ ⊂
=
Logical Inference
Common Sense Inference
In leaked audio, Clinton talks about Sanders supporters “living in parents’ basement”
![Page 15: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/15.jpg)
PrivatelyIn hacked fundraiser recording
In leaked recording⊂ ⊂
=
basement-dwellers
=
Logical Inference
Common Sense Inference
Stylistics
In leaked audio, Clinton talks about Sanders supporters “living in parents’ basement”
![Page 16: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/16.jpg)
PrivatelyIn hacked fundraiser recording
In leaked recording⊂ ⊂
=
basement-dwellers
=
Logical Inference
Common Sense Inference Stylistics
In leaked audio, Clinton talks about Sanders supporters “living in parents’ basement”
![Page 17: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/17.jpg)
Natural Language Inference
![Page 18: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/18.jpg)
Natural Language Inference(aka Recognizing Textual Entailment)
![Page 19: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/19.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
(aka Recognizing Textual Entailment)
![Page 20: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/20.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
(aka Recognizing Textual Entailment)
![Page 21: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/21.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
premise
hypothesis
(aka Recognizing Textual Entailment)
![Page 22: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/22.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
(aka Recognizing Textual Entailment)
p entails h if “typically, a human reading p would infer that h is most likely true.”
The Pascal Recognising Textual Entailment Challenge. Dagan et al. (2006)
![Page 23: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/23.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
Introduction
![Page 24: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/24.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
artist
composer
Introduction
![Page 25: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/25.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun CompositionAmerican composer
Introduction
![Page 26: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/26.jpg)
American composer
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
composer
Introduction
![Page 27: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/27.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
American composer
Charles Mingus
Introduction
![Page 28: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/28.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
Introduction
![Page 29: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/29.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
artist
composer
Introduction
![Page 30: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/30.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
![Page 31: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/31.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
lives in basement
is a basement-dwellerEquivalence
![Page 32: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/32.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
privatelyForward Entailment in leaked
audio
![Page 33: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/33.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
talks aboutReverse Entailment
slams
![Page 34: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/34.jpg)
Natural Language Inference
In leaked audio, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
Independent millennialsSanders supporters
![Page 35: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/35.jpg)
Natural Language Inference
At a press conference, Clinton talks about Sanders supporters living in basement
Hillary Clinton privately slams millennials as basement-dwellers
Exclusion privatelyat a press conference
![Page 36: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/36.jpg)
Equivalence x ⟺ y
Reverse Entailment x ⇒ y
Forward Entailment y ⇒ x
Independence x ⇏ y ⋀ y ⇏ x
Exclusion x ⇒ ¬y ⋀ y ⇒ ¬x
felinecat
animal
cat
cat pet
cat dog
animal
cat
![Page 37: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/37.jpg)
Lexical Semantics Resources
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
WordNet. Fellbaum (1998)
![Page 38: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/38.jpg)
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Lexical Semantics Resources
![Page 39: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/39.jpg)
Bilingual Pivoting
ahogados a la playa ...
get washed up on beaches ...
... fünf Landwirte , weil
... 5 farmers were in Ireland ...
...
oder wurden , gefoltert
or have been , tortured
festgenommen
thrown into jail
festgenommen
imprisoned
...
... ...
...
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Lexical Semantics Resources
Paraphrasing with bilingual parallel corpora. Bannard and Callison-Burch (2005)
![Page 40: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/40.jpg)
Vector Space Models
Bilingual Pivoting
ahogados a la playa ...
get washed up on beaches ...
... fünf Landwirte , weil
... 5 farmers were in Ireland ...
...
oder wurden , gefoltert
or have been , tortured
festgenommen
thrown into jail
festgenommen
imprisoned
...
... ...
...
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Lexical Semantics Resources
![Page 41: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/41.jpg)
Vector Space Models
Bilingual Pivoting
ahogados a la playa ...
get washed up on beaches ...
... fünf Landwirte , weil
... 5 farmers were in Ireland ...
...
oder wurden , gefoltert
or have been , tortured
festgenommen
thrown into jail
festgenommen
imprisoned
...
... ...
...
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
x ⇒ y ⋀ y ⇏ x
Lexical Semantics Resources
![Page 42: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/42.jpg)
Vector Space Models
Bilingual Pivoting
ahogados a la playa ...
get washed up on beaches ...
... fünf Landwirte , weil
... 5 farmers were in Ireland ...
...
oder wurden , gefoltert
or have been , tortured
festgenommen
thrown into jail
festgenommen
imprisoned
...
... ...
...
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
x shares some translation with y
Lexical Semantics Resources
![Page 43: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/43.jpg)
Vector Space Models
Bilingual Pivoting
ahogados a la playa ...
get washed up on beaches ...
... fünf Landwirte , weil
... 5 farmers were in Ireland ...
...
oder wurden , gefoltert
or have been , tortured
festgenommen
thrown into jail
festgenommen
imprisoned
...
... ...
...
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
x appears in similar contexts as y
Lexical Semantics Resources
![Page 44: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/44.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer
talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicate
talk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subject
talk about≈express
talk about≈see
talk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture talk about≈explain
talk about≈job
talk about≈issue
talk about≈relate
talk about≈sustain
talk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocate
talk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈givetalk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Lexical Semantics Resources
![Page 45: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/45.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer
talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicate
talk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subject
talk about≈express
talk about≈see
talk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture talk about≈explain
talk about≈job
talk about≈issue
talk about≈relate
talk about≈sustain
talk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocate
talk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈givetalk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Data-Driven ModelsPrecise but Small
Big but Noisy
Lexical Semantics Resources
![Page 46: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/46.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer
talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicate
talk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subject
talk about≈express
talk about≈see
talk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture talk about≈explain
talk about≈job
talk about≈issue
talk about≈relate
talk about≈sustain
talk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocate
talk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈givetalk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Data-Driven ModelsPrecise but Small
Big but Noisy
Lexical Semantics ResourcesCan we build lexical entailment resources automatically and at scale…
![Page 47: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/47.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer
talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicate
talk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subject
talk about≈express
talk about≈see
talk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture talk about≈explain
talk about≈job
talk about≈issue
talk about≈relate
talk about≈sustain
talk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocate
talk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈givetalk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
WordNet
act
communicate
address
harangue
rant
perform
practice
walk through scrimmage
relay
talk about
descant
Data-Driven ModelsPrecise but Small
Big but Noisy
Lexical Semantics ResourcesCan we build lexical entailment resources automatically and at scale…
…while maintaining WordNet-level
precision and interpretability?
![Page 48: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/48.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈seetalk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
The Paraphrase Database
PPDB: The Paraphrase Database. Ganitkevich et al. (2013)
![Page 49: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/49.jpg)
The Paraphrase Database
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈see talk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
Entailment
EquivalenceExclusion
Independent
![Page 50: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/50.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈seetalk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
The Paraphrase Database
![Page 51: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/51.jpg)
Distributional Signals of Semantics
![Page 52: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/52.jpg)
Distributional Signals of Semantics
Monolingual Contextual SimilaritiesLin and Pantel, 2001 (Alberta) Mikolov et al., 2013 (Google)
Pennington et al., 2014 (Stanford)
![Page 53: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/53.jpg)
Distributional Signals of Semantics
…converted from classical work to abstract expressionism after hearing Russian composer Igor
Stravinsky's "Rite of Spring”…
…South African contemporary artist, with abstract expressionism work featuring key aesthetics of the
most sought after artists…
Monolingual Contextual SimilaritiesLin and Pantel, 2001 (Alberta) Mikolov et al., 2013 (Google)
Pennington et al., 2014 (Stanford)
![Page 54: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/54.jpg)
Distributional Signals of Semantics
…converted from classical work to abstract expressionism after hearing Russian composer Igor
Stravinsky's "Rite of Spring”…
…South African contemporary artist, with abstract expressionism work featuring key aesthetics of the
most sought after artists…
Monolingual Contextual SimilaritiesLin and Pantel, 2001 (Alberta) Mikolov et al., 2013 (Google)
Pennington et al., 2014 (Stanford)
![Page 55: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/55.jpg)
Contextual Similarities
Stre
ngth
sWea
knes
ses
![Page 56: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/56.jpg)
Stre
ngth
sWea
knes
ses
dad/fathervs.
dad/lychee
Contextual Similarities
![Page 57: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/57.jpg)
Stre
ngth
sWea
knes
ses
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
Contextual Similarities
![Page 58: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/58.jpg)
Distributional Signals of Semantics
Bilingual Translational Similarity Bannard and Callison-Burch, 2005 (Edinburgh)
Kok and Brockett, 2010 (MSR) Ganitkevitch et al., 2013 (Hopkins)
![Page 59: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/59.jpg)
Distributional Signals of Semantics
…the directive include the extension to the period of
protection for composers…
…to favour the position of artists who have to travel
throughout the community…
…la directive comprennent la prolongation de la durée
de protection pour les artistes…
…favoriser la position des artistes qui doivent voyager à travers la communauté…
Bilingual Translational Similarity Bannard and Callison-Burch, 2005 (Edinburgh)
Kok and Brockett, 2010 (MSR) Ganitkevitch et al., 2013 (Hopkins)
![Page 60: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/60.jpg)
Distributional Signals of Semantics
…the directive include the extension to the period of
protection for composers…
…to favour the position of artists who have to travel
throughout the community…
…la directive comprennent la prolongation de la durée
de protection pour les artistes…
…favoriser la position des artistes qui doivent voyager à travers la communauté…
Bilingual Translational Similarity Bannard and Callison-Burch, 2005 (Edinburgh)
Kok and Brockett, 2010 (MSR) Ganitkevitch et al., 2013 (Hopkins)
![Page 61: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/61.jpg)
Bilingual TranslationsStre
ngth
sWea
knes
ses
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
Contextual Similarities
![Page 62: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/62.jpg)
Stre
ngth
sWea
knes
ses
Bilingual Translations
Contextual Similarities
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
dad/fathervs.
dad/mom
![Page 63: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/63.jpg)
Stre
ngth
sWea
knes
ses
Bilingual Translations
Contextual Similarities
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
dad/fathervs.
dad/mom
dad/parent vs.
dad/lychee
![Page 64: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/64.jpg)
Distributional Signals of Semantics
Lexico-Syntactic Patterns Hearst, 1992 (Berkeley)
Snow et al., 2006 (Stanford) Movshovitz-Attias and Cohen, 2015 (CMU)
![Page 65: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/65.jpg)
Distributional Signals of Semantics
How do composers and other artists survive and work in today's musical theatre scene?
As Luciano Berio did in his “Recital for Cathy”, creative artists such as composers, theatre
directors, choreographs, video artists or even circus ...
Lexico-Syntactic Patterns Hearst, 1992 (Berkeley)
Snow et al., 2006 (Stanford) Movshovitz-Attias and Cohen, 2015 (CMU)
![Page 66: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/66.jpg)
Distributional Signals of Semantics
How do composers and other artists survive and work in today's musical theatre scene?
As Luciano Berio did in his “Recital for Cathy”, creative artists such as composers, theatre directors, choreographs, video artists or even
circus ...
Lexico-Syntactic Patterns Hearst, 1992 (Berkeley)
Snow et al., 2006 (Stanford) Movshovitz-Attias and Cohen, 2015 (CMU)
![Page 67: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/67.jpg)
Stre
ngth
sWea
knes
ses
Bilingual Translations
Contextual Similarities
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
dad/fathervs.
dad/mom
dad/parent vs.
dad/lychee
Lexico-Syntactic Patterns
![Page 68: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/68.jpg)
Stre
ngth
sWea
knes
ses
Bilingual Translations
Contextual Similarities
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
dad/fathervs.
dad/mom
dad/parent vs.
dad/lychee
dad/parent vs.
dad/lychee
Lexico-Syntactic Patterns
![Page 69: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/69.jpg)
Stre
ngth
sWea
knes
ses
Bilingual Translations
Contextual Similarities
dad/fathervs.
dad/lychee
dad/fathervs.
dad/mom
dad/fathervs.
dad/mom
dad/parent vs.
dad/lychee
dad/parent vs.
dad/lychee
Lexico-Syntactic Patterns
dad/fathervs.
dad/lychee
![Page 70: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/70.jpg)
[ ]Lexico-Syntactic PatternsBilingual Translations
Logistic Regression
=
1 + e
.1
[ ] Contextual Similaritiesw1 w2 w3
[ ]P(equivalent) P(entailment) P(exclusion)
P(independent)
![Page 71: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/71.jpg)
[ ]Lexico-Syntactic PatternsBilingual Translations
Logistic Regression
=
1 + e
.1
[ ] Contextual Similaritiesw1 w2 w3
[ ]P(equivalent) P(entailment) P(exclusion)
P(independent)
Predict a probability distribution based over entailment relations…
![Page 72: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/72.jpg)
[ ]Lexico-Syntactic PatternsBilingual Translations
Logistic Regression
=
1 + e
.1
[ ] Contextual Similaritiesw1 w2 w3
[ ]P(equivalent) P(entailment) P(exclusion)
P(independent)
…based on all of the data-driven signals available.
![Page 73: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/73.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈seetalk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
The Paraphrase Database
![Page 74: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/74.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈seetalk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
The Paraphrase Database
![Page 75: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/75.jpg)
talk about≈speak
talk about≈discuss
talk about≈tell
talk about≈say
talk about≈mention
talk about≈about
talk about≈mean
talk about≈spoken
talk about≈address
talk about≈said
talk about≈discussion
talk about≈refer talk about≈talking
talk about≈raise
talk about≈alone
talk about≈debate
talk about≈chat
talk about≈told
talk about≈argued
talk about≈feel
talk about≈maintaintalk about≈comment
talk about≈make
talk about≈add
talk about≈approach
talk about≈causetalk about≈deliberations
talk about≈ask
talk about≈added
talk about≈tackletalk about≈bet
talk about≈betcha
talk about≈treat
talk about≈communicatetalk about≈described
talk about≈know
talk about≈to
talk about≈stated
talk about≈deal
talk about≈topic
talk about≈subjecttalk about≈express
talk about≈seetalk about≈highlighttalk about≈consider
talk about≈question
talk about≈touch
talk about≈sound
talk about≈noted
talk about≈nurture
talk about≈explain
talk about≈job
talk about≈issue
talk about≈relatetalk about≈sustaintalk about≈insert
talk about≈causing
talk about≈confront talk about≈time
talk about≈covered
talk about≈put
talk about≈will
talk about≈cite
talk about≈advocatetalk about≈indicate
talk about≈please
talk about≈regard
talk about≈hear
talk about≈kidding
talk about≈read
talk about≈dispute
talk about≈give
talk about≈say nothing
talk about≈say nothing of
talk about≈is done
talk about≈doesn’t say
talk about≈don’t speak
talk about≈doesn’t say
The Paraphrase Database
Can we build a resource like WordNet automatically, at scale,
and without loss of precision?
![Page 76: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/76.jpg)
Improving End-to-End RTE
p entails h if typically, a human reading p would infer that h is
most likely true.
![Page 77: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/77.jpg)
Improving End-to-End RTE
p entails h if typically, a human reading p would infer that h is
most likely true.
p = “A man is having a conversation.” h = “Some women are talking.”
![Page 78: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/78.jpg)
Improving End-to-End RTE
p entails h if typically, a human reading p would infer that h is
most likely true.
p = “A man is having a conversation.” h = “Some women are talking.”
No
![Page 79: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/79.jpg)
x1
man(x1)
x2 x3
patient(x2,x3) agent(x2,x1) have(x2) conversation(x3)
A man is having a conversation. Some woman are talking.
x1 x2
agent(x1,x2) talk(x1) woman(x2)
Improving End-to-End RTE
![Page 80: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/80.jpg)
x1
man(x1)
x2 x3
patient(x2,x3) agent(x2,x1) have(x2) conversation(x3)
A man is having a conversation. Some woman are talking.
x1 x2
agent(x1,x2) talk(x1) woman(x2)
∀x(man(x)⇒¬woman(x))
Improving End-to-End RTE
![Page 81: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/81.jpg)
x1
man(x1)
x2 x3
patient(x2,x3) agent(x2,x1) have(x2) conversation(x3)
A man is having a conversation. Some woman are talking.
x1 x2
agent(x1,x2) talk(x1) woman(x2)
∀x,h,c,t(have(h)⋀conversation(c)⋀talk(t) ⋀agent(h,x)⇒agent(t,x))
Improving End-to-End RTE
![Page 82: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/82.jpg)
0.0
0.2
0.4
0.5
0.7 0.66
0.49
No Axioms
Using PPDBPe
rform
ance
(F1
Scor
e)Improving End-to-End RTE
![Page 83: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/83.jpg)
0.0
0.2
0.4
0.5
0.7 0.660.61
0.49
No Axioms
Using PPDB
Using WordNetPe
rform
ance
(F1
Scor
e)Improving End-to-End RTE
![Page 84: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/84.jpg)
0.0
0.2
0.4
0.5
0.7 0.660.660.61
0.49
No Axioms
Using PPDB
Using WordNet
Human OraclePe
rform
ance
(F1
Scor
e)Improving End-to-End RTE
![Page 85: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/85.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
artist
composer
Introduction
![Page 86: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/86.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun CompositionAmerican composer
Introduction
![Page 87: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/87.jpg)
artist
Non-Compositional Semantics
![Page 88: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/88.jpg)
artist
composer
Non-Compositional Semantics
![Page 89: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/89.jpg)
artist
American composer composer
Non-Compositional Semantics
![Page 90: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/90.jpg)
artist
American composer 1950s American jazz composer
composer
Non-Compositional Semantics
![Page 91: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/91.jpg)
⟦modifier1 modifier2 … modifierk noun⟧
Non-Compositional Semantics
![Page 92: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/92.jpg)
O(NMk)
Non-Compositional Semantics
![Page 93: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/93.jpg)
American jazz composer
~270,000,000,000,000
O(NMk)
Non-Compositional Semantics
![Page 94: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/94.jpg)
American jazz composer
~270,000,000,000,000
O(NMk)
Non-Compositional Semantics
Problem #1: scalability
![Page 95: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/95.jpg)
Non-Compositional Semantics
“composer”
![Page 96: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/96.jpg)
Non-Compositional Semantics
“1950s American jazz composer”
![Page 97: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/97.jpg)
Non-Compositional Semantics
“1950s American jazz composer”
Problem #2: sparsity
![Page 98: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/98.jpg)
American composer
composer
Non-Compositional Semantics
![Page 99: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/99.jpg)
American composer
composer
Non-Compositional Semantics
American actor
actor
![Page 100: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/100.jpg)
American composer
composer
Non-Compositional Semantics
American actor
actor
American author
author
![Page 101: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/101.jpg)
American composer
composer
Non-Compositional Semantics
American actor
actor
American author
author
American singer
singer
?
? ? ?
?
![Page 102: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/102.jpg)
American composer
composer
Non-Compositional Semantics
American actor
actor
American author
author
American singer
singer
?
? ? ?
? Problem #3: generalizability
![Page 103: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/103.jpg)
American composer
composer
Compositional Semantics
![Page 104: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/104.jpg)
American composercomposer
Compositional Semantics
![Page 105: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/105.jpg)
American composercomposer
Compositional Semantics
American
![Page 106: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/106.jpg)
American composercomposer
Compositional Semantics
American
![Page 107: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/107.jpg)
American composercomposer
Compositional Semantics
Semantic Containment
![Page 108: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/108.jpg)
American composer
Compositional Semantics
American
Class-Instance Identification
composer
![Page 109: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/109.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun CompositionAmerican composer
Introduction
![Page 110: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/110.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
composer
American composer
Introduction
![Page 111: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/111.jpg)
Classes of Modifiers
American composer
composer
![Page 112: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/112.jpg)
Classes of Modifiers
Subsective
MH ⇒ H
American composer
composer
![Page 113: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/113.jpg)
Classes of Modifiers
Subsective
criminal
alleged criminal
Plain Non-Subsective
MH ⇒ H MH ⇏ H
American composer
composer
![Page 114: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/114.jpg)
Classes of Modifiers
Subsective
criminal
alleged criminal
Plain Non-Subsective
gun
Privative
fake gun
MH ⇒ ¬H MH ⇒ H MH ⇏ H
American composer
composer
![Page 115: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/115.jpg)
Equivalence MH ⟺ H It is her favorite book in the entire world.
Reverse Entailment
MH ⇒ H ⋀H ⇏ MH
She is an American composer.
Forward Entailment
MH ⇏ H ⋀ H ⇒ MH
She is the president’s potential
successor.
Independence MH ⇏ H ⋀ H ⇏ MH
She is the alleged hacker.
Exclusion MH ⇒ ¬H ⋀ H ⇒ ¬MH
She is a former senator.
![Page 116: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/116.jpg)
Eddy is a cat.
Natural Language Inference
![Page 117: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/117.jpg)
Eddy is a cat.
Eddy is a domestic cat.
Natural Language Inference
![Page 118: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/118.jpg)
Eddy is a cat.
Eddy is a domestic cat.
catdomestic
cat
Natural Language Inference
![Page 119: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/119.jpg)
Eddy is a cat.
Eddy is a domestic cat.
catdomestic
cat
Natural Language Inference
⇏
![Page 120: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/120.jpg)
Eddy is a domestic cat sitting on the ground looking out through a clear door screen.
⇏
Eddy is a cat sitting on the ground looking out through a clear door screen.
Natural Language Inference
![Page 121: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/121.jpg)
Eddy is a domestic cat sitting on the ground looking out through a clear door screen.
⇏
Eddy is a cat sitting on the ground looking out through a clear door screen.
p entails h if typically, a human reading p would infer that h is
most likely true.
Natural Language Inference
![Page 122: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/122.jpg)
Eddy is a domestic cat sitting on the ground looking out through a clear door screen.
Eddy is a cat sitting on the ground looking out through a clear door screen.
p entails h if typically, a human reading p would infer that h is
most likely true.
Natural Language Inference
⇒
![Page 123: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/123.jpg)
Natural Language Inference
Eddy is a domestic cat sitting on the ground looking out through a clear door screen.
Eddy is a cat sitting on the ground looking out through a clear door screen.
What types of inference rules
govern human inferences in practice?
p entails h if typically, a human reading p would infer that h is
most likely true.
⇒
![Page 124: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/124.jpg)
⇒
Natural Language Inference
p entails h if typically, a human reading p would infer that h is
most likely true.
Eddy is a domestic cat sitting on the ground looking out through a clear door screen.
Eddy is a cat sitting on the ground looking out through a clear door screen.
What, if any, generalizations can be
made to aide systems in performing natural language
inference?
What types of inference rules
govern human inferences in practice?
![Page 125: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/125.jpg)
Human Annotation of MH Compositions
![Page 126: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/126.jpg)
Human Annotation of MH Compositions
Eddy is a cat.
Eddy is a domestic cat.
H ⇒ MH?
![Page 127: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/127.jpg)
Human Annotation of MH Compositions
Eddy is a cat.
Eddy is a domestic cat.
MH ⇒ H?
![Page 128: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/128.jpg)
MH ⇒ H H ⇒ MH
Equiv. Yes Yes It is her favorite book in the entire world.
Rev. Ent. Yes Unk Eddy is a gray cat.
For. Ent. Unk Yes She is the president’s potential successor.
Indep. Unk Unk She is the alleged hacker.
Excl. No No She is a former senator.
![Page 129: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/129.jpg)
7%1%7%
62%
23%
EquivalenceReverse EntailmentIndependenceForward EntailmentExclusionUndefined
![Page 130: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/130.jpg)
7%1%7%
62%
23%
EquivalenceReverse EntailmentIndependenceForward EntailmentExclusionUndefined
normal subsective modifiers
![Page 131: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/131.jpg)
7%1%7%
62%
23%
EquivalenceReverse EntailmentIndependenceForward EntailmentExclusionUndefined
noun entails modifier?
![Page 132: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/132.jpg)
7%1%7%
62%
23%
EquivalenceReverse EntailmentIndependenceForward EntailmentExclusionUndefined
noun contradicts modifier?
![Page 133: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/133.jpg)
News
7%1%7%
62%
23%
Equivalence Reverse EntailmentIndependence Forward EntailmentExclusion Undefined
Images
4%2%
87%
7%
Literature
5%2%
66%
26%
Debate Forums
9%1%6%
53%
31%
![Page 134: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/134.jpg)
News
7%1%7%
62%
23%
Equivalence Reverse EntailmentIndependence Forward EntailmentExclusion Undefined
Images
4%2%
87%
7%
Literature
5%2%
66%
26%
Debate Forums
9%1%6%
53%
31%
H ⇒ MH?
![Page 135: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/135.jpg)
News
7%1%7%
62%
23%
Equivalence Reverse EntailmentIndependence Forward EntailmentExclusion Undefined
Images
4%2%
87%
7%
Literature
5%2%
66%
26%
Debate Forums
9%1%6%
53%
31%
The deadly attack killed at least 12
civilians. The new series will premiere in January.
A woman rides a bike on an outdoor trail
through a field.
H ⇒ MH?
![Page 136: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/136.jpg)
News
7%1%7%
62%
23%
Equivalence Reverse EntailmentIndependence Forward EntailmentExclusion Undefined
Images
4%2%
87%
7%
Literature
5%2%
66%
26%
Debate Forums
9%1%6%
53%
31%
I simply love the actual experience of being one with
the ocean and the life in it.
The entire bill is now subject to approval by the parliament.
H ⇒ MH?
Greenberg also was put under investigation for his
crucial role at the company.
![Page 137: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/137.jpg)
H ⇒ MH?
Entities are assumed to be real and relevant.
![Page 138: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/138.jpg)
H ⇒ MH?
Entities are assumed to be real and relevant.
![Page 139: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/139.jpg)
H ⇒ MH?
Entities are assumed to be prototypical.
Empirical Analysis
![Page 140: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/140.jpg)
News
7%1%7%
62%
23%
Equivalence Reverse EntailmentIndependence Forward EntailmentExclusion Undefined
Images
4%2%
87%
7%
Literature
5%2%
66%
26%
Debate Forums
9%1%6%
53%
31%
H ⇒ ¬MH?
![Page 141: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/141.jpg)
gun fake gun
H ⇒ ¬MH?
![Page 142: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/142.jpg)
H MH
H ⇒ ¬MH?
H ⇒ ¬MH MH ⇒ ¬H
![Page 143: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/143.jpg)
H MH
H ⇒ ¬MH?
H ⇒ ¬MH MH ⇒ ¬H
![Page 144: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/144.jpg)
Undefined Relations
![Page 145: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/145.jpg)
H
Undefined Relations
MH ⇒ H
MH
(like subsective)
![Page 146: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/146.jpg)
Undefined Relations
H ⇒ ¬MH(like privative)
H MH
![Page 147: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/147.jpg)
Equiv. Yes Yes It is her favorite book in the entire world.
Rev. Ent. Yes Unk Eddy is a gray cat.
For. Ent. Unk Yes She is the president’s potential successor.
Indep. Unk Unk She is the alleged hacker.
Excl. No No She is a former senator.
Undef. Yes No ?????
MH ⇒ H H ⇒ MH
![Page 148: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/148.jpg)
H ⇒ ¬MH
Undefined Relations
Bush travels Monday to Michigan to remark on the economy.
Bush travels Monday to Michigan to remark on the Japanese economy.
![Page 149: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/149.jpg)
Bush travels Monday to Michigan to remark on the Japanese economy.
MH ⇒ H
Undefined Relations
Bush travels Monday to Michigan to remark on the economy.
![Page 150: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/150.jpg)
criminal
alleged criminal
gun
fake gunAmerican composer
composer
Classes of Modifiers Revisited
MH ⇒ ¬H MH ⇒ H MH ⇏ H Subsective Plain Non-Subsective Privative
![Page 151: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/151.jpg)
Classes of Modifiers Revisited
50% 50%
Equivalence Reverse Entailment IndependenceForward Entailment Exclusion Undefined
100% 100%
MH ⇒ ¬H MH ⇒ H MH ⇏ H Subsective Plain Non-Subsective Privative
![Page 152: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/152.jpg)
5%14%
7%
54%
19%
Equivalence Reverse Entailment IndependenceForward Entailment Exclusion Undefined
4%1%
67%
28%37%
16% 1%16%
28%
3%
Classes of Modifiers Revisited
MH ⇒ ¬H MH ⇒ H MH ⇏ H Subsective Plain Non-Subsective Privative
![Page 153: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/153.jpg)
H ⇒ ¬MH
Privative Modifiers
Wilson signed off to pay the debts to the company.
Wilson signed off to pay the debts to the fictitious company.
![Page 154: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/154.jpg)
MH ⇒ H
Wilson signed off to pay the debts to the fictitious company.
Wilson signed off to pay the debts to the company.
Privative Modifiers
![Page 155: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/155.jpg)
5%14%
7%
54%
19%
Equivalence Reverse Entailment IndependenceForward Entailment Exclusion Undefined
4%1%
67%
28%37%
16% 1%16%
28%
3%
Classes of Modifiers Revisited
MH ⇒ ¬H MH ⇒ H MH ⇏ H Subsective Plain Non-Subsective Privative
![Page 156: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/156.jpg)
Classes of Modifiers Revisited
MH ⇒ ¬H MH ⇒ H MH ⇏ H Subsective Plain Non-Subsective Privative
Generalizations based on the class of the modifier lead to incorrect predictions more often than not.
![Page 157: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/157.jpg)
Modern Inference Systems
p entails h if typically, a human reading p would infer that h is
most likely true.
![Page 158: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/158.jpg)
Modern Inference Systems p = “The crowd roared.”
h = “The enthusiastic crowd roared.”
p entails h if typically, a human reading p would infer that h is
most likely true.
![Page 159: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/159.jpg)
Modern Inference Systems
Yes
p = “The crowd roared.” h = “The enthusiastic crowd roared.”
p entails h if typically, a human reading p would infer that h is
most likely true.
![Page 160: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/160.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Modern Inference Systems Ra
ndom
Gue
ssin
g
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
![Page 161: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/161.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
Partially proof-based
Modern Inference Systems
![Page 162: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/162.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
Partially proof-based
Supervised Learning
Modern Inference Systems
![Page 163: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/163.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
Partially proof-based
Supervised Learning
Deep Learning
Modern Inference Systems
![Page 164: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/164.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
Correct representation is difficult to capture explicitly and it is not currently being learned implicitly.
Modern Inference Systems
![Page 165: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/165.jpg)
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Accu
racy
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
Correct representation is difficult to capture explicitly and is currently not being learned implicitly.
Modern Inference Systems
![Page 166: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/166.jpg)
Discussion
![Page 167: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/167.jpg)
Discussion
The roared.crowd
![Page 168: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/168.jpg)
Discussion
The roared.
enthusiastic crowd
crowd
![Page 169: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/169.jpg)
Discussion
enthusiastic crowd
crowd
Set Containment
![Page 170: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/170.jpg)
Discussion
Set Containment
![Page 171: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/171.jpg)
DiscussionThe roared.crowd
![Page 172: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/172.jpg)
DiscussionThe ___ roared.crowd
P(enthusiastic) P(silent) P(imaginary)
Language Modeling
![Page 173: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/173.jpg)
Discussion
Word Sense Disambiguation
The roared.crowd
enthusiastic crowd
silent crowd
imaginary crowd
![Page 174: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/174.jpg)
DiscussionThe roared.crowd
Reference
![Page 175: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/175.jpg)
DiscussionThe roared.crowd
Reference
![Page 176: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/176.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
Reference
![Page 177: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/177.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
![Page 178: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/178.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
human
![Page 179: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/179.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
making noise human
![Page 180: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/180.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
![Page 181: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/181.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
excited/happy making noise
clapping yelling human
![Page 182: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/182.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
excited/happy making noise
clapping yelling human
![Page 183: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/183.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
excited/happy making noise
clapping yelling human
Assigning intrinsic
meaning to modifiers…
![Page 184: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/184.jpg)
DiscussionThe roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
excited/happy making noise
clapping yelling human
Determining whether they hold for individual
entities
![Page 185: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/185.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
composer
American composer
Introduction
![Page 186: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/186.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
American composer
Charles Mingus
Introduction
![Page 187: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/187.jpg)
American composercomposer
Compositional Semantics
American
![Page 188: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/188.jpg)
American composercomposer
Compositional Semantics
![Page 189: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/189.jpg)
American composercomposer
Compositional Semantics
![Page 190: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/190.jpg)
American composercomposer
Compositional SemanticsCan we assign intrinsic meaning to modifiers…
![Page 191: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/191.jpg)
American composercomposer
Compositional Semantics
…in such a way that we can we determine whether the modifier
holds for individual entities in practice?
Can we assign intrinsic meaning to modifiers…
![Page 192: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/192.jpg)
Step 1: Modifier InterpretationDetermine the properties entailed by the
modifier in the context of the head
![Page 193: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/193.jpg)
American jazz composerborn in America
influential in America prolific while in America
a product of America lived in America visited America
popular in America
Step 1: Modifier InterpretationDetermine the properties entailed by the
modifier in the context of the head
![Page 194: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/194.jpg)
born in America influential in America
prolific while in America a product of America
lived in America visited America
popular in America
Step 2: Class-Instance IdentificationDetermine, for a specific instance, whether
the necessary properties hold…Mingus's intricate, complex,
compositions in the genres of jazz and classical music illustrate his ability to be dynamic in both the
strings and the swing. Mingus truly was a product of America in all its historic complexities. His mother, Harriet, was half black and half
Chinese, and his father, Charles Sr., was half black and half Swedish, making Mingus a true reflection of
the hybrid nature of our divided nation…
American jazz composer
![Page 195: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/195.jpg)
Modifier InterpretationAmerican composer
![Page 196: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/196.jpg)
American composer
Modifier Interpretation
Americacomposer *
![Page 197: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/197.jpg)
American composer
Modifier Interpretation
Americacomposer *
composer from America composer born in America composer popular in America composer active in America
![Page 198: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/198.jpg)
American composer
Modifier Interpretation
Americacomposer *
⟨composer from America, 3702⟩ ⟨composer born in America, 1389⟩ ⟨composer popular in America, 1292⟩ ⟨composer active in America, 2041⟩
![Page 199: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/199.jpg)
American composer
Modifier Interpretation
Americacomposer *
⟨composer from America, 3702⟩ ⟨composer born in America, 1389⟩ ⟨composer popular in America, 1292⟩ ⟨composer active in America, 2041⟩
P(Y|X) = 11 + eXβ
![Page 200: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/200.jpg)
American composer
Modifier Interpretation
Americacomposer *
⟨composer from America, 0.93⟩ ⟨composer born in America, 0.94⟩ ⟨composer popular in America, 0.45⟩ ⟨composer active in America, 0.52⟩
P(Y|X) = 11 + eXβ
![Page 201: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/201.jpg)
American composer
Modifier Interpretation
Americacomposer *
⟨composer born in America, 0.94⟩ ⟨composer from America, 0.93⟩ ⟨composer active in America, 0.52⟩ ⟨composer popular in America, 0.45⟩
P(Y|X) = 11 + eXβ
![Page 202: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/202.jpg)
Modifier Interpretation
American composer born in America
American company based in America
American novel written in America
Produces good
results…
![Page 203: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/203.jpg)
Modifier Interpretation
child actor has child
risk manager takes risks
machine gun used by machine
…but not perfect.
![Page 204: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/204.jpg)
Class-Instance Identification
![Page 205: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/205.jpg)
Class-Instance Identification
American composer
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
Weighted modifier interpretations
Americacomposer *
![Page 206: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/206.jpg)
American composer
* is a composer
J.S. Bach Charles Mingus John Cage W.A. Mozart
Candidate instances
Class-Instance Identification
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
![Page 207: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/207.jpg)
“J.S. Bach born in America”
J.S. Bach Charles Mingus John Cage W.A. Mozart
Class-Instance Identification
American composer
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
Confidence = 0.94x21 + 0.93x34 + 0.52x329 + 0.45x4,043
![Page 208: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/208.jpg)
“J.S. Bach from America”
J.S. Bach Charles Mingus John Cage W.A. Mozart
Class-Instance Identification
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
American composer
Confidence = 0.94x21 + 0.93x34 + 0.52x329 + 0.45x4,043
![Page 209: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/209.jpg)
“J.S. Bach active in America”
J.S. Bach Charles Mingus John Cage W.A. Mozart
Class-Instance Identification
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
American composer
Confidence = 0.94x21 + 0.93x34 + 0.52x329 + 0.45x4,043
![Page 210: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/210.jpg)
“J.S. Bach popular in America”
J.S. Bach Charles Mingus John Cage W.A. Mozart
Confidence = 0.94x21 + 0.93x34 + 0.52x329 + 0.45x4,043
Class-Instance Identification
⟨___ born in America, 0.94⟩ ⟨___ from America, 0.93⟩ ⟨___ active in America, 0.52⟩ ⟨___ popular in America, 0.45⟩
American composer
![Page 211: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/211.jpg)
Class-Instance IdentificationAmerican composer jazz composer
JS Bach 0.21 0.04Charles Mingus 0.89 0.93John Cage 0.96 0.52WA Mozart 0.19 0.13Libby Larsen 0.72 0.24Duke Ellington 0.76 0.97Palestrina 0.04 0.03Ludwig van Beethoven 0.09 0.12Morton Feldman 0.88 0.31Frederick Chopin 0.33 0.32Barack Obama 0.14 0.35Herbie Hancock 0.62 0.95
![Page 212: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/212.jpg)
Class-Instance IdentificationAmerican jazz composer
JS Bach 0.25Charles Mingus 1.82John Cage 1.48WA Mozart 0.32Libby Larsen 0.96Duke Ellington 1.73Palestrina 0.07Ludwig van Beethoven 0.21Morton Feldman 1.19Frederick Chopin 0.65Barack Obama 0.49Herbie Hancock 1.57
![Page 213: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/213.jpg)
Class-Instance IdentificationAmerican jazz composer
Charles Mingus 1.82Duke Ellington 1.73Herbie Hancock 1.57John Cage 1.48Morton Feldman 1.19Libby Larsen 0.96Frederick Chopin 0.65Barack Obama 0.49WA Mozart 0.32JS Bach 0.25Ludwig van Beethoven 0.21Palestrina 0.07
![Page 214: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/214.jpg)
Reconstructing Wikipedia
![Page 215: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/215.jpg)
10
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
EACL 2017 Submission ***. Confidential review copy. DO NOT DISTRIBUTE.
(a) Uniform random sample. (b) Weighted random sample.
Figure 3: ROC curves for selected methods. Given a list of instances associated with confidence scores,ROC curves show the relationship between the number of true positives and the number of false positivesthat are retained by setting various threshold confidence values. The curve becomes linear once allremaining instances in the list have the same score (e.g., 0) as this makes it impossible to choose athreshold which adds true positives to the list without also including all remaining false positives.
sentences. Thus, it can provide non-zero scoresfor many more candidate instances. This enablesthe proposed methods to achieve a better trade-o↵ between extracting true positives versus falsepositives, than the baseline models do.
Uniform WeightedAUC Rec. AUC Recall
Baseline 0.55 0.23 0.53 0.28Hearst 0.56 0.03 0.52 0.02Hearst\ 0.57 0.04 0.53 0.02ModsH 0.68 0.08 0.60 0.06ModsI 0.71 0.09 0.65 0.09Hearst\+ModsH 0.70 0.09 0.61 0.08Hearst\+ModsI 0.73 0.10 0.66 0.10
Table 7: Recall of instances listed on Wikipediacategory pages. “Rec” is the recall against the en-tire set of instances appearing on the Wikipediapages. AUC captures the tradeo↵ between truepositives and false positives (see Figure 3).
7 Conclusion
We have presented an approach to IsA extractionwhich takes advantage of the compositionality ofnatural language. Existing approaches often treatclass labels as atomic units which must be observedin full in order to be populated with instances. Asa result, current methods are not able to handlethe infinite number of classes describable in natu-ral language, most of which never appear in text.Our method works by reasoning about each modi-fier in the label individually, in terms of the prop-erties that it implies about the instances. Thisapproach allows us to harness information that isspread across multiple sentences, and results in asignificant increase in the number of fine-grainedclasses which we are able to populate.
TODO: Add two or three lines of futurework.
TODO: Break some of the longer sen-tences containing “which” or “that”.
References
Anonymous. 2016. Unsupervised interpretation ofmultiple-modifier noun phrases. In submission.
Mohit Bansal, David Burkett, Gerard de Melo, andDan Klein. 2014. Structured learning for taxon-omy induction with belief propagation. In Pro-ceedings of the 52nd Annual Meeting of the Asso-ciation for Computational Linguistics (Volume1: Long Papers), pages 1041–1051, Baltimore,Maryland, June. Association for ComputationalLinguistics.
Marco Baroni and Roberto Zamparelli. 2010.Nouns are vectors, adjectives are matrices: Rep-resenting adjective-noun constructions in seman-tic space. In Proceedings of the 2010 Conferenceon Empirical Methods in Natural Language Pro-cessing, pages 1183–1193. Association for Com-putational Linguistics.
Lidong Bing, Sneha Chaudhari, Richard Wang,andWilliam Cohen. 2015. Improving distant su-pervision for information extraction using labelpropagation through lists. In Proceedings of the2015 Conference on Empirical Methods in Natu-ral Language Processing, pages 524–529, Lisbon,Portugal, September. Association for Computa-tional Linguistics.
Kurt Bollacker, Colin Evans, Praveen Paritosh,Tim Sturge, and Jamie Taylor. 2008. Free-base: a collaboratively created graph databasefor structuring human knowledge. In Proceed-ings of the 2008 ACM SIGMOD internationalconference on Management of data, pages 1247–1250. ACM.
E. Choi, T. Kwiatkowski, and L. Zettlemoyer.2015. Scalable semantic parsing with partial
Reconstructing Wikipedia
![Page 216: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/216.jpg)
10
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
EACL 2017 Submission ***. Confidential review copy. DO NOT DISTRIBUTE.
(a) Uniform random sample. (b) Weighted random sample.
Figure 3: ROC curves for selected methods. Given a list of instances associated with confidence scores,ROC curves show the relationship between the number of true positives and the number of false positivesthat are retained by setting various threshold confidence values. The curve becomes linear once allremaining instances in the list have the same score (e.g., 0) as this makes it impossible to choose athreshold which adds true positives to the list without also including all remaining false positives.
sentences. Thus, it can provide non-zero scoresfor many more candidate instances. This enablesthe proposed methods to achieve a better trade-o↵ between extracting true positives versus falsepositives, than the baseline models do.
Uniform WeightedAUC Rec. AUC Recall
Baseline 0.55 0.23 0.53 0.28Hearst 0.56 0.03 0.52 0.02Hearst\ 0.57 0.04 0.53 0.02ModsH 0.68 0.08 0.60 0.06ModsI 0.71 0.09 0.65 0.09Hearst\+ModsH 0.70 0.09 0.61 0.08Hearst\+ModsI 0.73 0.10 0.66 0.10
Table 7: Recall of instances listed on Wikipediacategory pages. “Rec” is the recall against the en-tire set of instances appearing on the Wikipediapages. AUC captures the tradeo↵ between truepositives and false positives (see Figure 3).
7 Conclusion
We have presented an approach to IsA extractionwhich takes advantage of the compositionality ofnatural language. Existing approaches often treatclass labels as atomic units which must be observedin full in order to be populated with instances. Asa result, current methods are not able to handlethe infinite number of classes describable in natu-ral language, most of which never appear in text.Our method works by reasoning about each modi-fier in the label individually, in terms of the prop-erties that it implies about the instances. Thisapproach allows us to harness information that isspread across multiple sentences, and results in asignificant increase in the number of fine-grainedclasses which we are able to populate.
TODO: Add two or three lines of futurework.
TODO: Break some of the longer sen-tences containing “which” or “that”.
References
Anonymous. 2016. Unsupervised interpretation ofmultiple-modifier noun phrases. In submission.
Mohit Bansal, David Burkett, Gerard de Melo, andDan Klein. 2014. Structured learning for taxon-omy induction with belief propagation. In Pro-ceedings of the 52nd Annual Meeting of the Asso-ciation for Computational Linguistics (Volume1: Long Papers), pages 1041–1051, Baltimore,Maryland, June. Association for ComputationalLinguistics.
Marco Baroni and Roberto Zamparelli. 2010.Nouns are vectors, adjectives are matrices: Rep-resenting adjective-noun constructions in seman-tic space. In Proceedings of the 2010 Conferenceon Empirical Methods in Natural Language Pro-cessing, pages 1183–1193. Association for Com-putational Linguistics.
Lidong Bing, Sneha Chaudhari, Richard Wang,andWilliam Cohen. 2015. Improving distant su-pervision for information extraction using labelpropagation through lists. In Proceedings of the2015 Conference on Empirical Methods in Natu-ral Language Processing, pages 524–529, Lisbon,Portugal, September. Association for Computa-tional Linguistics.
Kurt Bollacker, Colin Evans, Praveen Paritosh,Tim Sturge, and Jamie Taylor. 2008. Free-base: a collaboratively created graph databasefor structuring human knowledge. In Proceed-ings of the 2008 ACM SIGMOD internationalconference on Management of data, pages 1247–1250. ACM.
E. Choi, T. Kwiatkowski, and L. Zettlemoyer.2015. Scalable semantic parsing with partial
Reconstructing Wikipedia
Best Existing Non-Compositional Method
(Lexico-Syntactic Patterns) AUC = 0.57
![Page 217: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/217.jpg)
10
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
EACL 2017 Submission ***. Confidential review copy. DO NOT DISTRIBUTE.
(a) Uniform random sample. (b) Weighted random sample.
Figure 3: ROC curves for selected methods. Given a list of instances associated with confidence scores,ROC curves show the relationship between the number of true positives and the number of false positivesthat are retained by setting various threshold confidence values. The curve becomes linear once allremaining instances in the list have the same score (e.g., 0) as this makes it impossible to choose athreshold which adds true positives to the list without also including all remaining false positives.
sentences. Thus, it can provide non-zero scoresfor many more candidate instances. This enablesthe proposed methods to achieve a better trade-o↵ between extracting true positives versus falsepositives, than the baseline models do.
Uniform WeightedAUC Rec. AUC Recall
Baseline 0.55 0.23 0.53 0.28Hearst 0.56 0.03 0.52 0.02Hearst\ 0.57 0.04 0.53 0.02ModsH 0.68 0.08 0.60 0.06ModsI 0.71 0.09 0.65 0.09Hearst\+ModsH 0.70 0.09 0.61 0.08Hearst\+ModsI 0.73 0.10 0.66 0.10
Table 7: Recall of instances listed on Wikipediacategory pages. “Rec” is the recall against the en-tire set of instances appearing on the Wikipediapages. AUC captures the tradeo↵ between truepositives and false positives (see Figure 3).
7 Conclusion
We have presented an approach to IsA extractionwhich takes advantage of the compositionality ofnatural language. Existing approaches often treatclass labels as atomic units which must be observedin full in order to be populated with instances. Asa result, current methods are not able to handlethe infinite number of classes describable in natu-ral language, most of which never appear in text.Our method works by reasoning about each modi-fier in the label individually, in terms of the prop-erties that it implies about the instances. Thisapproach allows us to harness information that isspread across multiple sentences, and results in asignificant increase in the number of fine-grainedclasses which we are able to populate.
TODO: Add two or three lines of futurework.
TODO: Break some of the longer sen-tences containing “which” or “that”.
References
Anonymous. 2016. Unsupervised interpretation ofmultiple-modifier noun phrases. In submission.
Mohit Bansal, David Burkett, Gerard de Melo, andDan Klein. 2014. Structured learning for taxon-omy induction with belief propagation. In Pro-ceedings of the 52nd Annual Meeting of the Asso-ciation for Computational Linguistics (Volume1: Long Papers), pages 1041–1051, Baltimore,Maryland, June. Association for ComputationalLinguistics.
Marco Baroni and Roberto Zamparelli. 2010.Nouns are vectors, adjectives are matrices: Rep-resenting adjective-noun constructions in seman-tic space. In Proceedings of the 2010 Conferenceon Empirical Methods in Natural Language Pro-cessing, pages 1183–1193. Association for Com-putational Linguistics.
Lidong Bing, Sneha Chaudhari, Richard Wang,andWilliam Cohen. 2015. Improving distant su-pervision for information extraction using labelpropagation through lists. In Proceedings of the2015 Conference on Empirical Methods in Natu-ral Language Processing, pages 524–529, Lisbon,Portugal, September. Association for Computa-tional Linguistics.
Kurt Bollacker, Colin Evans, Praveen Paritosh,Tim Sturge, and Jamie Taylor. 2008. Free-base: a collaboratively created graph databasefor structuring human knowledge. In Proceed-ings of the 2008 ACM SIGMOD internationalconference on Management of data, pages 1247–1250. ACM.
E. Choi, T. Kwiatkowski, and L. Zettlemoyer.2015. Scalable semantic parsing with partial
Reconstructing WikipediaBest Proposed Compositional Method
AUC = 0.73
Best Existing Non-Compositional Method
(Lexico-Syntactic Patterns) AUC = 0.57
![Page 218: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/218.jpg)
Lexical Entailment
Semantic Containment
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
American composer
Charles Mingus
Introduction
![Page 219: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/219.jpg)
Lexical Entailment
Semantic Containment
Introduction
Summary and Future Work
Class-Instance Identification
Adding Semantics to Data-Driven Paraphrasing. Pavlick et al. ACL (2015)
Compositional Entailment in Adjective Nouns. Pavlick and Callison-Burch. ACL (2016) So-Called Non-Subsective Adjectives. Pavlick and Callison-Burch. *SEM (2016)
Fine-Grained Class Extraction via Modifier Composition. Pavlick and Pasca. ACL (2017)
Modifier-Noun Composition
![Page 220: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/220.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
![Page 221: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/221.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
Equivalence Reverse Entailment
Forward Entailment Independent Exclusion
![Page 222: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/222.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
![Page 223: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/223.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.61
0.49
No Axioms
Using PPDB
Using WordNet
Human Oracle
![Page 224: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/224.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
![Page 225: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/225.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
![Page 226: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/226.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
Plain Non-Subsective
5%14%
7%
54%
19%
Subsective
4%1%
67%
28%
Privative
37%
16% 1%16%
28%
3%
![Page 227: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/227.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
![Page 228: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/228.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
20
36
52
68
84
100
86.886.687.386.685.38685.385.3
Rand
om G
uess
ing
Tran
sfor
mat
ion-
base
d St
ern
and
Dag
an (2
012)
Bag
of W
ords
Logi
stic
Reg
ress
ion
Mag
nini
et a
l. (2
014)
Bag
of V
ecto
rs
RNN
LSTM
LSTM
+ T
rans
fer
Bow
man
et a
l. (2
015)
![Page 229: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/229.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
![Page 230: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/230.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
![Page 231: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/231.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
composersAmerican
composers
![Page 232: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/232.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
![Page 233: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/233.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
10
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
EACL 2017 Submission ***. Confidential review copy. DO NOT DISTRIBUTE.
(a) Uniform random sample. (b) Weighted random sample.
Figure 3: ROC curves for selected methods. Given a list of instances associated with confidence scores,ROC curves show the relationship between the number of true positives and the number of false positivesthat are retained by setting various threshold confidence values. The curve becomes linear once allremaining instances in the list have the same score (e.g., 0) as this makes it impossible to choose athreshold which adds true positives to the list without also including all remaining false positives.
sentences. Thus, it can provide non-zero scoresfor many more candidate instances. This enablesthe proposed methods to achieve a better trade-o↵ between extracting true positives versus falsepositives, than the baseline models do.
Uniform WeightedAUC Rec. AUC Recall
Baseline 0.55 0.23 0.53 0.28Hearst 0.56 0.03 0.52 0.02Hearst\ 0.57 0.04 0.53 0.02ModsH 0.68 0.08 0.60 0.06ModsI 0.71 0.09 0.65 0.09Hearst\+ModsH 0.70 0.09 0.61 0.08Hearst\+ModsI 0.73 0.10 0.66 0.10
Table 7: Recall of instances listed on Wikipediacategory pages. “Rec” is the recall against the en-tire set of instances appearing on the Wikipediapages. AUC captures the tradeo↵ between truepositives and false positives (see Figure 3).
7 Conclusion
We have presented an approach to IsA extractionwhich takes advantage of the compositionality ofnatural language. Existing approaches often treatclass labels as atomic units which must be observedin full in order to be populated with instances. Asa result, current methods are not able to handlethe infinite number of classes describable in natu-ral language, most of which never appear in text.Our method works by reasoning about each modi-fier in the label individually, in terms of the prop-erties that it implies about the instances. Thisapproach allows us to harness information that isspread across multiple sentences, and results in asignificant increase in the number of fine-grainedclasses which we are able to populate.
TODO: Add two or three lines of futurework.
TODO: Break some of the longer sen-tences containing “which” or “that”.
References
Anonymous. 2016. Unsupervised interpretation ofmultiple-modifier noun phrases. In submission.
Mohit Bansal, David Burkett, Gerard de Melo, andDan Klein. 2014. Structured learning for taxon-omy induction with belief propagation. In Pro-ceedings of the 52nd Annual Meeting of the Asso-ciation for Computational Linguistics (Volume1: Long Papers), pages 1041–1051, Baltimore,Maryland, June. Association for ComputationalLinguistics.
Marco Baroni and Roberto Zamparelli. 2010.Nouns are vectors, adjectives are matrices: Rep-resenting adjective-noun constructions in seman-tic space. In Proceedings of the 2010 Conferenceon Empirical Methods in Natural Language Pro-cessing, pages 1183–1193. Association for Com-putational Linguistics.
Lidong Bing, Sneha Chaudhari, Richard Wang,andWilliam Cohen. 2015. Improving distant su-pervision for information extraction using labelpropagation through lists. In Proceedings of the2015 Conference on Empirical Methods in Natu-ral Language Processing, pages 524–529, Lisbon,Portugal, September. Association for Computa-tional Linguistics.
Kurt Bollacker, Colin Evans, Praveen Paritosh,Tim Sturge, and Jamie Taylor. 2008. Free-base: a collaboratively created graph databasefor structuring human knowledge. In Proceed-ings of the 2008 ACM SIGMOD internationalconference on Management of data, pages 1247–1250. ACM.
E. Choi, T. Kwiatkowski, and L. Zettlemoyer.2015. Scalable semantic parsing with partial
![Page 234: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/234.jpg)
Lexical Entailment
Semantic Containment
Class-Instance Identification
0.0
0.2
0.4
0.5
0.7 0.660.660.610.49
2036526884
100
10
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
EACL 2017 Submission ***. Confidential review copy. DO NOT DISTRIBUTE.
(a) Uniform random sample. (b) Weighted random sample.
Figure 3: ROC curves for selected methods. Given a list of instances associated with confidence scores,ROC curves show the relationship between the number of true positives and the number of false positivesthat are retained by setting various threshold confidence values. The curve becomes linear once allremaining instances in the list have the same score (e.g., 0) as this makes it impossible to choose athreshold which adds true positives to the list without also including all remaining false positives.
sentences. Thus, it can provide non-zero scoresfor many more candidate instances. This enablesthe proposed methods to achieve a better trade-o↵ between extracting true positives versus falsepositives, than the baseline models do.
Uniform WeightedAUC Rec. AUC Recall
Baseline 0.55 0.23 0.53 0.28Hearst 0.56 0.03 0.52 0.02Hearst\ 0.57 0.04 0.53 0.02ModsH 0.68 0.08 0.60 0.06ModsI 0.71 0.09 0.65 0.09Hearst\+ModsH 0.70 0.09 0.61 0.08Hearst\+ModsI 0.73 0.10 0.66 0.10
Table 7: Recall of instances listed on Wikipediacategory pages. “Rec” is the recall against the en-tire set of instances appearing on the Wikipediapages. AUC captures the tradeo↵ between truepositives and false positives (see Figure 3).
7 Conclusion
We have presented an approach to IsA extractionwhich takes advantage of the compositionality ofnatural language. Existing approaches often treatclass labels as atomic units which must be observedin full in order to be populated with instances. Asa result, current methods are not able to handlethe infinite number of classes describable in natu-ral language, most of which never appear in text.Our method works by reasoning about each modi-fier in the label individually, in terms of the prop-erties that it implies about the instances. Thisapproach allows us to harness information that isspread across multiple sentences, and results in asignificant increase in the number of fine-grainedclasses which we are able to populate.
TODO: Add two or three lines of futurework.
TODO: Break some of the longer sen-tences containing “which” or “that”.
References
Anonymous. 2016. Unsupervised interpretation ofmultiple-modifier noun phrases. In submission.
Mohit Bansal, David Burkett, Gerard de Melo, andDan Klein. 2014. Structured learning for taxon-omy induction with belief propagation. In Pro-ceedings of the 52nd Annual Meeting of the Asso-ciation for Computational Linguistics (Volume1: Long Papers), pages 1041–1051, Baltimore,Maryland, June. Association for ComputationalLinguistics.
Marco Baroni and Roberto Zamparelli. 2010.Nouns are vectors, adjectives are matrices: Rep-resenting adjective-noun constructions in seman-tic space. In Proceedings of the 2010 Conferenceon Empirical Methods in Natural Language Pro-cessing, pages 1183–1193. Association for Com-putational Linguistics.
Lidong Bing, Sneha Chaudhari, Richard Wang,andWilliam Cohen. 2015. Improving distant su-pervision for information extraction using labelpropagation through lists. In Proceedings of the2015 Conference on Empirical Methods in Natu-ral Language Processing, pages 524–529, Lisbon,Portugal, September. Association for Computa-tional Linguistics.
Kurt Bollacker, Colin Evans, Praveen Paritosh,Tim Sturge, and Jamie Taylor. 2008. Free-base: a collaboratively created graph databasefor structuring human knowledge. In Proceed-ings of the 2008 ACM SIGMOD internationalconference on Management of data, pages 1247–1250. ACM.
E. Choi, T. Kwiatkowski, and L. Zettlemoyer.2015. Scalable semantic parsing with partial
![Page 235: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/235.jpg)
Future Directions
![Page 236: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/236.jpg)
The roared.crowd
enthusiastic crowd
real
humanmaking noise
excited/happy
excited/happy making noise
clapping yelling human
Future Directions
![Page 237: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/237.jpg)
Future DirectionsThe roared.crowd
real
humanmaking noise
excited/happy
![Page 238: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/238.jpg)
Future DirectionsThe roared.crowd
real
humanmaking noise
happy
enthusiastic
![Page 239: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/239.jpg)
Future DirectionsThe red circle.
![Page 240: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/240.jpg)
Future Directions“common
sense knowledge”
![Page 241: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/241.jpg)
Future Directions“common
sense knowledge”What is it?
World Knowledge? Pragmatics?
How do we represent it?
Distributional? Symbolic? Triple stores? Probability distributions?
How is it learned?Is it distributional?
Is text enough?
When/how is it accessed?What can be
precomputed? What happens at
“runtime”?
![Page 242: Compositional Lexical Semantics for Natural Language Inference · Compositional Lexical Semantics for Natural Language Inference Thesis Defense Ellie Pavlick Department of Computer](https://reader033.fdocuments.us/reader033/viewer/2022042711/5f79f93482a1bd055b0d8876/html5/thumbnails/242.jpg)
Thank you! Questions!