Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional...
Transcript of Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional...
![Page 1: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/1.jpg)
Session 2Session 2
b i h i l l f i l• basic theoretical tools of computational linguistics– logic: lambda calculus– typed feature structuresyp
• Natural Language Analysis– problems with context‐free grammars– problems with context‐free grammars– fixing the problemsfi i th bl d b th fi i– fixing the problems caused by the fixes: moving towards mild context‐sensitivity with CCGs
![Page 2: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/2.jpg)
Computational LinguisticsComputational Linguistics
computational modelneuro‐psych / cognitive models
computational model
computational implementation
line of arbitrariness
computational implementation
![Page 3: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/3.jpg)
Basic computational modelling toolsBasic computational modelling tools
• basic computational linguistic tools forbasic computational linguistic tools for representing semantics– logic– event‐based semantics
• generation• analysis– event‐based semantics
– ontologies– lambda expressions
analysis
• basic computational linguistic tools for representing informationp g– feature structures– unification
![Page 4: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/4.jpg)
Refresher : Logical ExpressionsRefresher : Logical Expressions
chase (x, y) run (x) run (y)
Some combination of di t d l i lpredicates and logical
connectors
![Page 5: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/5.jpg)
Refresher : Logical ExpressionsRefresher : Logical Expressions
bodychase (x, y) run (x) run (y)x y
Then you bind thei bl ith i tvariables with appropriate
quantifiersq
![Page 6: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/6.jpg)
Refresher : Logical ExpressionsRefresher : Logical Expressions
chase (x, y) run (x) run (y)x y
All th i bl th t iAll the variables that occur in the body of the formula, are on
.free)
y ,usually bound with a quantifier at the front!!!
FOPC
(n
quantifier at the front!!!
![Page 7: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/7.jpg)
Lambda ExpressionsLambda ExpressionsWe can turn expressions into functions pusing lambda expressions
chase (x, y)λx λy
λ chase (x, y)λx λy (farmer, duckling)
chase (farmer, duckling)
![Page 8: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/8.jpg)
Lambda ExpressionsLambda ExpressionsWe turn expressions into e tu e p ess o s tofunctions by using lambda expressions
chase (x, y)λx λy
λ chase (x, y)λx λy (farmer)
chase (farmer, y)λy
![Page 9: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/9.jpg)
Unification
![Page 10: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/10.jpg)
Example: UnificationExample: Unification
cat: Scat: Ssubj: cat: N
⊔verb: cat: V
verb: cat: Vverb: cat: Vnum: pl
= ?= ?
![Page 11: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/11.jpg)
Example: UnificationExample: Unification
cat: Ssubj: cat: N
t Vverb: cat: Vnum: plp
![Page 12: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/12.jpg)
Example: UnificationExample: Unification
cat: Scat: Ssubj: cat: N
⊔num: #1
verb: cat: Vverb: cat: Vnum: #1 verb: cat: V
num: pl
= ?
![Page 13: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/13.jpg)
Example: UnificationExample: Unification
cat: Sbjsubj: cat: N
num: pl
verb:
p
cat: Vverb: cat: Vnum: pl
![Page 14: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/14.jpg)
Example: UnificationExample: Unification
cat: Scat: Ssubj: ⊔
cat: Nbj Nnum: #1 subj: cat: N
num: sgverb: cat: V
g
⊔num: #1 ⊔verb: cat: Vverb: cat: V
num: pl
= ?
![Page 15: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/15.jpg)
Typed UnificationTyped Unification
• Nowadays grammars usually work with typed unification– distinct information types are defined as having particular combinations of attributesparticular combinations of attributes
– types are organised into inheritance lattices
![Page 16: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/16.jpg)
Example: Typed UnificationExample: Typed Unification
cat: Sbjsubj: cat: N
num: plnominal
verb:
p
cat: Vverb: cat: Vnum: pl
lverbal
clause
![Page 17: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/17.jpg)
Example: Typed UnificationExample: Typed Unification
t Scat: Ssubj: cat: N
num: plnominal
verb: cat: Vnum: pl
clauseverbal
num: sg⊔ = ?nominal
g⊔ ?
![Page 18: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/18.jpg)
Example: Typed UnificationExample: Typed Unification
t Scat: Ssubj: cat: N
num: pl
verb: cat: Vnum: pl
num: sg⊔ = ?⊔ ?
![Page 19: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/19.jpg)
Feature GeometriesFeature Geometries
• Much of modern linguistics is now expressed as bundles of features
• and how these are distributed aroundsyntactic structuressyntactic structures
• Some special kinds of features flow along the `backbone’ provided by the tree structures:backbone provided by the tree structures: head features
![Page 20: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/20.jpg)
Basic computational modelling toolsBasic computational modelling tools
• basic computational linguistic tools for• basic computational linguistic tools for representing information– feature structures– unification
• basic computational linguistic tools for• basic computational linguistic tools for representing semantics– event‐based semantics– ontologies– description logics– lambda expressionslambda expressions
![Page 21: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/21.jpg)
Grammar andGrammar and Feature Unification
![Page 22: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/22.jpg)
Problems with CF Phrase Structure Grammars
• Difficult to capture dependencies between constituentsconstituents– the boy runs
– the boys run
– * the boy run
– * the boys runs
![Page 23: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/23.jpg)
Problems with CF Phrase Structure Grammars
• Difficult to capture dependencies between constituentsconstituents– the boy opens the door
– *?? the boy opens
– the boy sits
– *?? the boy sits the table
![Page 24: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/24.jpg)
CF SolutionCF Solution
• exploding the number of rules is one way to provide a solution…p
S NPsing VPsingS NPplural VPplural
Detsing {a, this, the}Detplural {some these the}S NPplural VPplural
NP NPsingNP NPpluralNPsing Detsing Nsing
Detplural {some, these, the}
Nsing {boy, girl, …}Nplural {boys, girls, …}g g g
NPplural Detplural Nplural
VPsing Vintsing
p { y , g , }
Vintsing {sits, …}Vintplural {sit, …}g g
VPplural Vintplural
VPsing Vtrsing NP
p
Vtrsing {opens, …}Vtrplural {open, …}
VPplural Vtrplural NP
grammarlexicon
![Page 25: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/25.jpg)
A better solutionA better solution…
h ll i h• What we really want to say is that some constituents share properties
The boy runsthe ‘Subject’ and the ‘Verb’ agree in number
i e they share the same value for their number– i.e., they share the same value for their number feature
![Page 26: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/26.jpg)
Phrase structure rules with featuresPhrase structure rules with features
S → NP VP+singular +singular
VP → V (NP)VP → V (NP)+singular+singular gg
![Page 27: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/27.jpg)
Phrase structure rules with featuresPhrase structure rules with features
S → NP VP+plural +plural
VP → V (NP)VP → V (NP)+plural+plural pp
![Page 28: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/28.jpg)
Phrase structure rules with featuresPhrase structure rules with features
S → NP VP[ number: sing ] [ number: sing]
VP → V (NP)VP → V (NP)[ number: sing ][ number: sing] [ g ][ g]
![Page 29: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/29.jpg)
Phrase structure rules with featuresPhrase structure rules with features
S → NP VP[ number: pl ] [ number: pl]
VP → V (NP)VP → V (NP)[ number: pl ][ number: pl] [ p ][ p ]
![Page 30: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/30.jpg)
Generalised RulesGeneralised Rules
S → NP VP[ number: x ] [ number: x]
VP → V (NP)VP → V (NP)[ number: x ][ number: x] [ ][ ]
![Page 31: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/31.jpg)
Generalised Rules (PATR formalism)
S → NP VP<NP number> = <VP number>
VP → V (NP)VP → V (NP)<VP number> = <V number>
Grammar = {PSGrammar = {PS‐‐rules + rules + path equationspath equations}}
![Page 32: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/32.jpg)
Features Feature structuresAttribute‐value matrices (AVMs)
+singular [number: singular]
+ing‐form [verb: ing‐form]
+masc gender: masc+masc gender: masc+sing number: sing
Feature structuresUnification
CompatibilityInformation
grammar, phonology, …
![Page 33: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/33.jpg)
Susan laughsg
S → NP VP
[Subj: Susan[Subj: #1Actor: #1] S Pred: laughs
Actor: Susan]
Actor: #1]
Actor: Susan]
NP : [Subj: Susan] VP : [Pred: laughsNP : [Subj: Susan] VP : [Pred: laughsActor: X]
psg6.grm psg6.grm
![Page 34: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/34.jpg)
Feature RepresentationFeature Representation
i b l• Syntactic tree becomes a more complex structure
• Each node in the tree is in fact a bundle of features
• Particular rules (specified in the grammar) specify what conditions hold on the featurespecify what conditions hold on the feature structuresU ll l l i diti h ld• Usually: local – i.e., conditions hold over a dominating node and its children
![Page 35: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/35.jpg)
Final moveFinal move…
• All information is moved into the feature structure – even the tree structure…– HPSG (Head‐driven Phrase Structure Grammar)(Head driven Phrase Structure Grammar)
Head: XHead: XDaughters: <Y Z …>Daughters: <Y Z …>
XX
ggCat: CCat: CZZYY
![Page 36: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/36.jpg)
Summary: RecapSummary: Recap
• This is really the current state of the art in computational linguistics: all linguistic p g ginformation is represented as feature bundles
• This is called a ‘information based’ paradigm• This is called a information‐based paradigm
![Page 37: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/37.jpg)
Example Unification GrammarsExample Unification Grammars
PATR-II : WinPatr System ...
![Page 38: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/38.jpg)
Example GrammarExample Grammar
Parameter start symbol is SRule {Satz}
S -> NP VP.
Rule {VP intransitiv}VP -> Vi.
R l {VP t iti }Rule {NP-Name}
NP -> Name.
Rule {VP transitiv}VP -> Vt NP.
Rule {VP transitiv mit PP}Rule {NP}
NP -> Det N.
Rule {VP transitiv mit PP}VP -> Vt NP PP.
Rule {VP ditransitiv}Rule {VP ditransitiv}VP -> Vt2 NP NP.
Rule {VP mit PP-Objekt}Rule {VP mit PP Objekt}VP -> Vpo PP.
Rule {einfache PP}PATRPATR--formalismformalism
Rule {einfache PP}PP -> P NP.
![Page 39: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/39.jpg)
Example LexiconExample Lexicon\w cried \w girl \\w thew the\w cried\c Vi
\w saw
\w girl\c N
\w student
\\w thew the\\c Detc Det
\\w aw a\w saw\c Vt
\w gave
\w student\c N
\w book
\\w aw a\\c Detc Det
\\w inw in\w gave\c Vt2
\w book\c N
\\w inw in\\c Pc P
\\w onw on\\w onw on\\c Pc P
PATRPATR--formalismformalismf03b.grmf03b.grmf02.lexf02.lex
![Page 40: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/40.jpg)
Example Lexicon (with features)Example Lexicon (with features)\w cried \w girl \\w thew the\w cried\c Vi
\w saw
\w girl\c N\f sg
\\w thew the\\c Detc Det
\\w aw a\w saw\c Vt
\w gave
\w student\c N\f sg
\\w aw a\\c Detc Det
\\w inw in\w gave\c Vt2
\w sings
\f sg
\w book\c N
\\w inw in\\c Pc P
\\w onw on\w sings\c Vi\f 3sg
\c N\f sg
\w girls
\\w onw on\\c Pc P
g\c N\f pl
\w students\c N\f pl
![Page 41: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/41.jpg)
Example Lexicon (with features)Example Lexicon (with features)\w cried \w girl \\w thew the\w cried\c Vi
\w saw
\w girl\c N\f sg
\\w thew the\\c Detc Det
\\w aw a\w saw\c Vt
\w gave
\w student\c N\f sg
\\w aw a\\c Detc Det
\\w inw in [num: sg][num: sg]\w gave\c Vt2
\w sings
\f sg
\w book\c N
\\w inw in\\c Pc P
\\w onw onLet sg beLet sg be [num[num
[num: sg].[num: sg].
\w sings\c Vi\f 3sg
\c N\f sg
\w girls
\\w onw on\\c Pc P [num: sg[num: sg
pers: 3].pers: 3].g\c N\f pl
Let 3sg beLet 3sg be sg [psg [p
p ]p ]
\w students\c N\f pl
![Page 42: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/42.jpg)
Example Lexicon (with features)Example Lexicon (with features)\w cried \w girl \\w thew the\w cried\c Vi
\w saw
\w girl\c N\f sg
\\w thew the\\c Detc Det
\\w aw a\w saw\c Vt
\w gave
\w student\c N\f sg
\\w aw a\\c Detc Det
\\w inw in
[lex: student][lex: student][cat: N ].[cat: N ].\w gave
\c Vt2
\w sings
\f sg
\w book\c N
\\w inw in\\c Pc P
\\w onw on
[ ][ ]
[lex: sings][lex: sings]\w sings\c Vi\f 3sg
\c N\f sg
\w girls
\\w onw on\\c Pc P
[lex: sings][lex: sings][cat: V ].[cat: V ].
g\c N\f pl
\w students\c N\f pl
![Page 43: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/43.jpg)
Head features are usually ‘passed up’ to the dominating node
S → NP VPNP h d VP h dNP h d VP h d<NP head num> = <VP head num><NP head num> = <VP head num>
<NP head pers> = <VP head pers><NP head pers> = <VP head pers>
![Page 44: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/44.jpg)
Head features are usually ‘passed up’ to the dominating node
R lR l {VP i t iti }{VP i t iti }RuleRule {VP intransitiv}{VP intransitiv}
VP VP → V:V:<VP head> = <V><VP head> = <V><V subcat> = i.<V subcat> = i.
Let V beLet V be [cat: V].[cat: V].\w sings\c Vi\f 3sg
[lex: sings][lex: sings][cat: V ].[cat: V ].
Let Vi be V [subcat: i].Let Vi be V [subcat: i].
[cat: V[cat: Vg [cat: V[cat: Vsubcat: i]subcat: i]
![Page 45: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/45.jpg)
exercise with the example grammar…
what structure (both tree structure and feature structure) does the grammar produce for:) g p
“the boys sing”the boys singand what would it do for
“the boy sing”y g
![Page 46: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/46.jpg)
Feature‐based Parsing: “the boys sing”
S:S:[ t S[ t S
1:1:S S
[ cat: S[ cat: Ssubj: [ cat: NPsubj: [ cat: NP
spec: [ cat: Detspec: [ cat: Detlex: thelex: thenum: pl ]num: pl ]
_____|___________|______NP VP NP VP | || | num: pl ]num: pl ]
head: [ cat: Nhead: [ cat: Nlex: boyslex: boysnum: plnum: plpers: 3 ] ]pers: 3 ] ]
___|____ |___|____ |Det N V Det N V the boys sing the boys sing
pers: 3 ] ]pers: 3 ] ]pred: [ cat: VPpred: [ cat: VP
head: [ cat: Vhead: [ cat: Vsubcat:isubcat:ilex: singlex: singlex: singlex: singnum: plnum: plpers: 3 ] ] ]pers: 3 ] ] ]
1 parse found1 parse foundpa se ou dpa se ou d
![Page 47: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/47.jpg)
Problem of surface interpretationProblem of surface interpretation
• If we interpret a syntactic tree as a set of instructions for forming a semantic ginterpretation, then we do not need to rely on a ‘deep’ structure for meaninga deep structure for meaning…
![Page 48: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/48.jpg)
How to interpret the ‘surface’ structure?: C iti l S ti F ti A li tiCompositional Semantics as Function Application
“John went”S : went (John)
NP J h VP ( )NP : John VP : went (__)
λx [ go´ (x) ]J λx.[ go (x) ]J
![Page 49: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/49.jpg)
Semantic Interpretation RuleSemantic Interpretation Rule
S function composition
NP VP
λx.[ VERB´ (x) ]X
![Page 50: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/50.jpg)
Semantic composition represented i.t.o. Semantic features
S : e (__,__)x
NP : x VP : e
S → NP VPS → NP VP[ sem: e (x , _) ] [ sem: x ] [ sem: e]
![Page 51: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/51.jpg)
Semantic featuresSemantic features
S → NP VPS → NP VPS → NP VP
[ sem: e (x , _) ] [ sem: x ] [ sem: e]<S sem event> = <VP sem event><S sem event> = <VP sem event><S sem actor> = <S subj sem><S sem actor> = <S subj sem><S subj> = <NP><S subj> = <NP>
![Page 52: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/52.jpg)
Semantic featuresSemantic features
S → NP VP•• as informal feature valuesas informal feature values
S → NP VP[ sem: e (x , _) ] [ sem: x ] [ sem: e]
•• as path equationsas path equations
S → NP VP<S sem event> = <VP sem event><S sem event> = <VP sem event><S sem event> = <VP sem event><S sem event> = <VP sem event><S sem actor> = <S subj sem><S sem actor> = <S subj sem><S subj> = <NP><S subj> = <NP>jj
![Page 53: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/53.jpg)
Surface StructureSurface Structure
• If we had a surface structure, we could interpret it…p
• But we don’t…
H d h h• How can we produce the structures that natural human languages use without invoking far too powerful mechanisms?
![Page 54: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/54.jpg)
Remaining problemsRemaining problems…
T f i i l T i i l d h• Transformations are in general Turing equivalent and hence unlikely for a language model
• Seems implausible that our language facility evolved as aSeems implausible that our language facility evolved as a mechanism with that kind of power
• Unification is also potentially a very expensive operation• Need strong constraint for just which structures are to be
unified with othersf h h b k h• One way of achieving this is by making the syntactic
backbone do more work…
![Page 55: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/55.jpg)
A different style of grammarA different style of grammar
• Combinatory Categorical Grammar (CCG)– generalisation of Categorial Grammars
• Ajdukiewicz (1935)j ( )“Die syntaktische Konnexität”. Studia Philosophica, 1:1‐27.
• Bar‐Hillel (1953)“A quasi‐arithmetic notation for syntactic description”. Language 29:47‐58.
promoted particularly for computational linguistics by Mark Steedman
![Page 56: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/56.jpg)
Combinatory Categorical GrammarCCG
• In a CCG all lexical items are given categories that state their potential for combination with pother lexical (and non‐lexical) units
• There are only general rules on how to• There are only general rules on how to combine categories, no particular `grammar’ lrules.
![Page 57: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/57.jpg)
Analysis with CCGAnalysis with CCG
Example: “leave the room”
collect the lexical items and their categories (given in the lexicon):
leave :‐ s/np
h /the :‐ np/n
room :‐ nroom : n
![Page 58: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/58.jpg)
Analysis with CCGAnalysis with CCG
Example: “leave the room”
leave :‐ s/np
the : np/n‘the’ is a unit looking for an ‘n’ on its right to combine with tothe :‐ np/n
room :‐ n
on its right to combine with to give an ‘np’
room : n
![Page 59: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/59.jpg)
Analysis with CCGAnalysis with CCG
leave :‐ s/np
the : np/nthe :‐ np/n
room :‐ nroom : n
![Page 60: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/60.jpg)
Analysis with CCGAnalysis with CCG
leave :‐ s/np
the : np/nthe :‐ np/n
room :‐ nthe room :‐ np
room : n
“forward application”
• (>) the room : np
forward application
• (>) the room :‐ np
![Page 61: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/61.jpg)
Analysis with CCGAnalysis with CCG
l hleave
np/n nthe room
s/np p/
np>
/ p
np
![Page 62: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/62.jpg)
Analysis with CCGAnalysis with CCG
leave :‐ s/np
the room :‐ np
![Page 63: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/63.jpg)
Analysis with CCGAnalysis with CCG
leave :‐ s/npthe room :‐ np
• leave the room :‐ sp
![Page 64: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/64.jpg)
Analysis with CCGAnalysis with CCG
l hleave
np/n nthe room
s/np p/
np>
/ p
np
s>
s
![Page 65: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/65.jpg)
Most basic combination rulesMost basic combination rules
• Functional Application
⇒–X/Y Y ⇒ X
⇒ forward application (>)–Y X\Y ⇒ X forward application (>)
backward application (<)
![Page 66: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/66.jpg)
CCG: analysisCCG: analysis
“Keats eats apples”
Keats: NKeats: Napples: N
eats: (S\N) / N
![Page 67: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/67.jpg)
CCG: analysisCCG: analysis
Keats: NPapples: NP
eats:(S\NP)/NP
>
<
St dSteedman
![Page 68: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/68.jpg)
CCG: analysisCCG: analysis
Keats: NPapples: NP
eats:(S\NP)/NP
>
<
St dSteedman
![Page 69: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/69.jpg)
More combination rulesMore combination rules
• Coordination
⇒–X CONJ X ⇒ X
![Page 70: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/70.jpg)
CCG: analysisCCG: analysis
St dSteedman
![Page 71: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/71.jpg)
CCG: analysisCCG: analysis
St dSteedman
![Page 72: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/72.jpg)
Important features of CCGImportant features of CCG
• Highly lexicalized grammarsO l l i l i i CCG– Only lexical items in CCG
– Analysis is strongly restricted
• Construct syntactic and semantic representations synchronouslyrepresentations synchronously
![Page 73: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/73.jpg)
Semantics (traditional)Semantics (traditional)
• eat := (s / npnom) \ npacc( / pnom) \ pacc
λ λ ( )• λx2 λx1 . eat (x1 , x2 )
![Page 74: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/74.jpg)
CCG: analysis with semanticsCCG: analysis with semantics
λx2 λx1 . eat (x1 , x2 )
Keats: NPapples: NP aeats:(S\NP)/NP
>
<
St dSteedman
![Page 75: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/75.jpg)
CCG: analysis with semanticsCCG: analysis with semantics
functional application
(S \ NP) / NP NP S \ NP
λ λ ( )λx2 λx1 . eat (x1 , x2 ) a
λx1 . eat (x1 , a )
![Page 76: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/76.jpg)
An interesting pointAn interesting point
• The syntactic structure is not itself important
• It is just a record of the instructions that wereIt is just a record of the instructions that were carried out to build the correct semantics
![Page 77: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/77.jpg)
Important features of CCGImportant features of CCG
• Highly lexicalized grammars– Only lexical items in CCGOnly lexical items in CCG– Analysis is strongly restricted
• Construct syntactic and semantic• Construct syntactic and semantic representations synchronously
• Highly constrained computational complexitycomplexity
![Page 78: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/78.jpg)
Complexity classesComplexity classes
• Subclasses of polynomial time
– Linear time: O(n)Linear time: O(n)
– Quadratic time: O(n2)
– Cubic time: O(n3) CCG: O(n6)Cubic time: O(n )
• Exponential time: O (kn)
• Non‐deterministic polynomial time : NP
( )
• Non‐deterministic polynomial time : NP
n: length of input k: difficulty of problem
![Page 79: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/79.jpg)
‘Chomsky’ HierarchyChomsky Hierarchy
Type 0 : unrestricted
Type 1 : context‐sensitive
TRANSFORMATIONAL GRAMMAR
Mildly‐context sensitive
Type 2 : context‐free
Type 3 : regular
![Page 80: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/80.jpg)
Features of CCG: summaryFeatures of CCG: summary
• CCG is one of the most quickly developing grammatical formalisms for linguistic processing developing at this time
I fl ibili ll h d l l• Its flexibility appears well matched to real language use, including sentence fragments and difficult combinations
• Its computational complexity is low enough for realIts computational complexity is low enough for real processing
• Semantic interpretation is provided at the same time as syntactic analysis.
![Page 81: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/81.jpg)
Computational Use of CCGComputational Use of CCG
• OpenCCG
– White, Baldridge, Kruijff, …
– freely accessible
• Grammars now using a particular kind of semantics:
– Hybrid‐Logic Dependency Semantics
![Page 82: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/82.jpg)
Hybrid Logic Dependency SemanticsHybrid‐Logic Dependency Semantics
complex category HLDS expressionp g ydefinition
combining categories, feature structures, and semantic
HLDS expression
thingstructures, and semantic
constraints
flower
sg
X
![Page 83: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/83.jpg)
Similarity between HLDS and feature structures
Baldridge / Kruijff
![Page 84: Session 2 - uni-bremen.de · Session 2 • bibasic thilheoretical tools of computatilional linguistics – logic: lambda calculus – typyped feature structures • Natural Language](https://reader030.fdocuments.us/reader030/viewer/2022041222/5e0cc3dda949245c2a2586d5/html5/thumbnails/84.jpg)
Basic computational modelling toolsBasic computational modelling tools
• basic computational linguistic tools forbasic computational linguistic tools for representing semantics– logic– event‐based semantics
• generation• analysis– event‐based semantics
– ontologies– lambda expressions
analysis
• basic computational linguistic tools for representing informationp g– feature structures– unification