Post on 12-Jul-2020
Linguists-defined vs. Machine-inducedNatural Language Structures for
Executable Semantic Parsing
Siva ReddyStanford NLP Group
1
Mirella Lapata Mark Steedman Oscar Tackstrom
Tom Kwiatkowski Michael Collins Slav Petrov
Dipanjan Das Jianpeng Cheng Vijay Saraswat
2
Semantic Parsing
Natural Language (NL)
Machine Executable Representation (MR)P
arser
3
Semantic Parsing
NL
Leonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.
MR cast(TITANIC,DICAPRIO,JACK)∧
director(TITANIC,CAMERON)
TRUE
Parser
KB
Execute
7.7/10 · IMDb
88% · Rotten Tomatoes
James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More
Initial release: November 18, 1997 (London)
Director: James Cameron
Featured song: My Heart Will Go On
Cast
1997 ‧ Drama film/Romance ‧ 3h 30m
Titanic
LeonardoDiCaprioJack Dawson
KateWinsletRose DeWittBukater
Billy ZaneCaledonHockley
GloriaStuartRose DeWittBukater
Kathy BatesMolly Brown
Feedback
Edinburgh - From your Internet address - Use precise location - Learn more
Help Send feedback Privacy Terms
titanic 1997 Sign in
titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1
1 of 1 29/05/15 20:39
4
Semantic ParsingN
LLeonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.
MR cast(TITANIC,DICAPRIO,JACK)∧
director(TITANIC,CAMERON)
TRUE
Parser
KB
Execute
7.7/10 · IMDb
88% · Rotten Tomatoes
James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More
Initial release: November 18, 1997 (London)
Director: James Cameron
Featured song: My Heart Will Go On
Cast
1997 ‧ Drama film/Romance ‧ 3h 30m
Titanic
LeonardoDiCaprioJack Dawson
KateWinsletRose DeWittBukater
Billy ZaneCaledonHockley
GloriaStuartRose DeWittBukater
Kathy BatesMolly Brown
Feedback
Edinburgh - From your Internet address - Use precise location - Learn more
Help Send feedback Privacy Terms
titanic 1997 Sign in
titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1
1 of 1 29/05/15 20:39
4
Semantic ParsingN
LLeonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.
MR cast(TITANIC,DICAPRIO,JACK)∧
director(TITANIC,CAMERON)
TRUE
Parser
KB
Execute
7.7/10 · IMDb
88% · Rotten Tomatoes
James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More
Initial release: November 18, 1997 (London)
Director: James Cameron
Featured song: My Heart Will Go On
Cast
1997 ‧ Drama film/Romance ‧ 3h 30m
Titanic
LeonardoDiCaprioJack Dawson
KateWinsletRose DeWittBukater
Billy ZaneCaledonHockley
GloriaStuartRose DeWittBukater
Kathy BatesMolly Brown
Feedback
Edinburgh - From your Internet address - Use precise location - Learn more
Help Send feedback Privacy Terms
titanic 1997 Sign in
titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1
1 of 1 29/05/15 20:39
4
Semantic ParsingN
LLeonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.
MR cast(TITANIC,DICAPRIO,JACK)∧
director(TITANIC,CAMERON)
TRUE
Parser
KB
Execute
7.7/10 · IMDb
88% · Rotten Tomatoes
James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More
Initial release: November 18, 1997 (London)
Director: James Cameron
Featured song: My Heart Will Go On
Cast
1997 ‧ Drama film/Romance ‧ 3h 30m
Titanic
LeonardoDiCaprioJack Dawson
KateWinsletRose DeWittBukater
Billy ZaneCaledonHockley
GloriaStuartRose DeWittBukater
Kathy BatesMolly Brown
Feedback
Edinburgh - From your Internet address - Use precise location - Learn more
Help Send feedback Privacy Terms
titanic 1997 Sign in
titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1
1 of 1 29/05/15 20:39
4
Application: Question Answering
Feedback
James CameronTitanic, Director
James Francis Cameron is a Canadian filmmaker, inventor, engineer, philanthropist, anddeep-sea explorer who has directed the two biggest box office films of all time. He firstfound major success with the science-fiction hit The Terminator. Wikipedia
Movies and overview
James Cameron - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/James_CameronHe then became a popular Hollywood director and was hired to write and direct ..... in
the TV documentary special Titanic: The Final Word with James Cameron, ...James Cameron filmography - Suzy Amis - The Abyss - Gale Anne Hurd
Titanic (1997 film) - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/Titanic_(1997_film)Titanic is a 1997 American epic romantic disaster film directed, written, co-produced, and
co-edited by James Cameron.Gloria Stuart - Heart of the Ocean - Danny Nucci - J. Bruce Ismay
James Cameron - IMDbwww.imdb.com/name/nm0000116/James Cameron, Writer: Aliens. ... He landed his first professional film job as art
director, miniature-set builder, and process-projection ... 1997 Titanic (written by).
James Cameron - Biography - Director, Producer ...www.biography.com/people/james-cameron-54657017 Mar 2015 - Oscar-winning director James Cameron is best known for the highly
acclaimed, box-office hits Aliens (1986), Titanic (1997) and Avatar (2009).
'Titanic' director denies Jack and Rose could have survived ...www.nme.com/filmandtv/news/-titanic-director-denies...-/28658614 Sep 2012 - Titanic director James Cameron has dismissed an internet rumour
claiming that the movie's doomed lovers Jack and Rose could both have ...
Biography: Titanic and Avatar Director James Cameron ...www.watchmojo.com › Videos › Film › MoviesBiography: Titanic and Avatar Director James Cameron He's the creative genius
behind cinema's most iconic films, such as The Terminator, The Abyss and ...
Who was the director of Titanic? - Quorawww.quora.com › Movie Business and Industry › Movie ProductionTitanic (1997 movie): Why didn't Jack climb onto the plank with Rose when there ...
Titanic (1997 movie): James Cameron more or less copied the line "he exists ...
'Titanic' Director James Cameron Speaks about Lasting ...www.spiegel.de › English Site › Zeitgeist › History6 Apr 2012 - Canadian film director James Cameron has made three films about the
Titantic , including the 1997 blockbuster of the same name.
Titanic - Cast, Crew, Director and Awards - NYTimes.comwww.nytimes.com/movies/movie/158894/Titanic/detailsFull Acting Credits for Titanic » ... Director - James Cameron ... Best Director - James
[di/wr] Cameron - 1997 Broadcast Film Critics Association; Best Director ...
'Titanic' and 'Avatar' director James Cameron reaches ocean'slatimesblogs.latimes.com/.../titanic-and-avatar-director-james-cameron-re...
Freebase Question Answering
who is the director of titanic Sign in
who is the director of titanic - Google Search https://www.google.co.uk/search?client=ubuntu&channel=fs&q=who+is+the+director+of+titanic&ie=utf-8&oe=utf-8&gfe_rd=cr&ei=gtNoVd71M...
1 of 2 29/05/15 22:25
5
Application: Question Answering
Feedback
James CameronTitanic, Director
James Francis Cameron is a Canadian filmmaker, inventor, engineer, philanthropist, anddeep-sea explorer who has directed the two biggest box office films of all time. He firstfound major success with the science-fiction hit The Terminator. Wikipedia
Movies and overview
James Cameron - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/James_CameronHe then became a popular Hollywood director and was hired to write and direct ..... in
the TV documentary special Titanic: The Final Word with James Cameron, ...James Cameron filmography - Suzy Amis - The Abyss - Gale Anne Hurd
Titanic (1997 film) - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/Titanic_(1997_film)Titanic is a 1997 American epic romantic disaster film directed, written, co-produced, and
co-edited by James Cameron.Gloria Stuart - Heart of the Ocean - Danny Nucci - J. Bruce Ismay
James Cameron - IMDbwww.imdb.com/name/nm0000116/James Cameron, Writer: Aliens. ... He landed his first professional film job as art
director, miniature-set builder, and process-projection ... 1997 Titanic (written by).
James Cameron - Biography - Director, Producer ...www.biography.com/people/james-cameron-54657017 Mar 2015 - Oscar-winning director James Cameron is best known for the highly
acclaimed, box-office hits Aliens (1986), Titanic (1997) and Avatar (2009).
'Titanic' director denies Jack and Rose could have survived ...www.nme.com/filmandtv/news/-titanic-director-denies...-/28658614 Sep 2012 - Titanic director James Cameron has dismissed an internet rumour
claiming that the movie's doomed lovers Jack and Rose could both have ...
Biography: Titanic and Avatar Director James Cameron ...www.watchmojo.com › Videos › Film › MoviesBiography: Titanic and Avatar Director James Cameron He's the creative genius
behind cinema's most iconic films, such as The Terminator, The Abyss and ...
Who was the director of Titanic? - Quorawww.quora.com › Movie Business and Industry › Movie ProductionTitanic (1997 movie): Why didn't Jack climb onto the plank with Rose when there ...
Titanic (1997 movie): James Cameron more or less copied the line "he exists ...
'Titanic' Director James Cameron Speaks about Lasting ...www.spiegel.de › English Site › Zeitgeist › History6 Apr 2012 - Canadian film director James Cameron has made three films about the
Titantic , including the 1997 blockbuster of the same name.
Titanic - Cast, Crew, Director and Awards - NYTimes.comwww.nytimes.com/movies/movie/158894/Titanic/detailsFull Acting Credits for Titanic » ... Director - James Cameron ... Best Director - James
[di/wr] Cameron - 1997 Broadcast Film Critics Association; Best Director ...
'Titanic' and 'Avatar' director James Cameron reaches ocean'slatimesblogs.latimes.com/.../titanic-and-avatar-director-james-cameron-re...
Freebase Question Answering
who is the director of titanic Sign in
who is the director of titanic - Google Search https://www.google.co.uk/search?client=ubuntu&channel=fs&q=who+is+the+director+of+titanic&ie=utf-8&oe=utf-8&gfe_rd=cr&ei=gtNoVd71M...
1 of 2 29/05/15 22:25
5
Other Semantic Parsing Applications
6
This Talk
NaturalLanguage
MachineRepresentation
vs.
NaturalLanguage Syntax Machine
Representation
SyntacticParser
vs.
NaturalLanguage
Domain-specificNL Structure
MachineRepresentation
MachineInduced
7
This Talk
NaturalLanguage
MachineRepresentation
vs.
NaturalLanguage Syntax Machine
Representation
SyntacticParser
vs.
NaturalLanguage
Domain-specificNL Structure
MachineRepresentation
MachineInduced
7
This Talk
NaturalLanguage
MachineRepresentation
vs.
NaturalLanguage Syntax Machine
Representation
SyntacticParser
vs.
NaturalLanguage
Domain-specificNL Structure
MachineRepresentation
MachineInduced
7
Outline
1. Semantic Parsing with Linguists-defined NL Structures
2. Semantic Parsing with Machine-induced NL Structures
8
Semantic Parsing with Linguists-defined NL Structures
Reddy, Lapata, Steedman (TACL 2014)Reddy, Tackstrom, Collins, Kwiatkowski, Das, Steedman, Lapata (TACL 2016)Reddy, Tackstrom, Petrov, Steedman, Lapata (EMNLP 2017)
9
Motivation
NaturalLanguage Syntax Machine
Representation
SyntacticParser
1. Decades of research on syntax and parsing
2. Syntax is task-agnostic and universal
3. Annotation does not require target-domain experts
10
Motivation
NaturalLanguage Syntax Machine
Representation
SyntacticParser
1. Semantic Parsing data is expensive
2. Many treebanks
3. Constrained search
11
End-to-End Semantic Parsing[Zelle & Mooney, 1996; Zettlemoyer & Collins, 2005; Kwiatkowski et al., 2010; Liang et al., 2011;Artzi & Zettlemoyer, 2011; Krishnamurthy & Mitchell, 2012; Berant et al., 2013; Pasupat & Liang, 2015;
Yih et al., 2015]
Grammar learning problem
I director→ N : λx.film.director(x)
I of→ (NP\NP)/NP :λ f λgλx.∃y∃e. f (y)∧g(x)∧film.directed by(e)∧arg1(e,y)∧arg2(e,x)
Question
Who is the director of Titanic?
Grounded Logical Form
λx.∃e. film.director(x) ∧film.directed by(e) ∧
arg1(e,Titanic)∧ arg2(e,x)
12
Intermediate Semantic Parsing[Kwiatkowski et al., 2013; Reddy et al., 2014; Choi et al., 2015; Artzi et al., 2015]
Language toungrounded logical form
Ungrounded logical form togrounded logical form
Question
Who is the director of Titanic?
Ungrounded Logical Form
λx.TARGET(xa)∧ director(xa)∧director event(xe) ∧
arg0(xe,xa)∧ arg.of(xe,Titanic)
Grounded Logical Form
λx.∃e. film.director(x) ∧film.directed by(e) ∧
arg2(e,x)∧ arg1(e,Titanic)
13
Task Setting
Training Data: Question and Answer Pairs
Evaluation: Question Answering on Freebase
Resources: Syntactic Parser, Syntax to Ungrounded Logic
14
Freebase is a Graphusa
natashaobama
p q USpresident
r s
ColumbiaUniversity
mBarackObama
nMichelleObama
m m n n
education.university
Bachelorof Arts
1992
headquarters
.country
headquarters
.organisation
person.
nationality.arg2
person.
nationality.arg1
person.
parents.arg2
person.
parents.arg1
perso
n.
paren
ts.arg2
perso
n.
paren
ts.arg1
education.institution
education.student
marriage.spouse
marriage.spouse
education
.institution
education.degree ed
ucation
.deg
ree
education
.studen
t
marriage
.from
marriage
.spouse
marriage
.from
marriage
.spouse
type
type
15
Freebase is a Graphusa
natashaobama
p q USpresident
r s
ColumbiaUniversity
mBarackObama
nMichelleObama
m m n n
education.university
Bachelorof Arts
1992
headquarters
.country
headquarters
.organisation
person.
nationality.arg2
person.
nationality.arg1
person.
parents.arg2
person.
parents.arg1
perso
n.
paren
ts.arg2
perso
n.
paren
ts.arg1
education.institution
education.student
marriage.spouse
marriage.spouse
education
.institution
education.degree ed
ucation
.deg
ree
education
.studen
t
marriage
.from
marriage
.spouse
marriage
.from
marriage
.spouse
type
type
15
Freebase is a Graphusa
natashaobama
p q USpresident
r s
ColumbiaUniversity
mBarackObama
nMichelleObama
m m n n
education.university
Bachelorof Arts
1992
headquarters
.country
headquarters
.organisation
person.
nationality.arg2
person.
nationality.arg1
person.
parents.arg2
person.
parents.arg1
perso
n.
paren
ts.arg2
perso
n.
paren
ts.arg1
education.institution
education.student
marriage.spouse
marriage.spouse
education
.institution
education.degree ed
ucation
.deg
ree
education
.studen
t
marriage
.from
marriage
.spouse
marriage
.from
marriage
.spouse
type
type
15
Freebase is a Graphusa
natashaobama
p q USpresident
r s
ColumbiaUniversity
mBarackObama
nMichelleObama
m m n n
education.university
Bachelorof Arts
1992
headquarters
.country
headquarters
.organisation
person.
nationality.arg2
person.
nationality.arg1
person.
parents.arg2
person.
parents.arg1
perso
n.
paren
ts.arg2
perso
n.
paren
ts.arg1
education.institution
education.student
marriage.spouse
marriage.spouse
education
.institution
education.degree ed
ucation
.deg
ree
education
.studen
t
marriage
.from
marriage
.spouse
marriage
.from
marriage
.spouse
type
type
15
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
16
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
16
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
16
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
16
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning Model
SyntacticParser Syntax to Logic
Graph Matching
16
Our Approach
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
16
Logical Form to Ungrounded GraphCameron directed Titanic in 1997
λe.directed.arg1(e,Cameron)∧directed.arg2(e,Titanic)∧directed.in(e,1997)
Titanic
Cameron
1997
17
Logical Form to Ungrounded GraphCameron directed Titanic in 1997
λe.directed.arg1(e,Cameron)∧directed.arg2(e,Titanic)∧directed.in(e,1997)
Titanic
e
Cameron e
e
1997
17
Logical Form to Ungrounded GraphCameron directed Titanic in 1997
λe.directed.arg1(e,Cameron)∧directed.arg2(e,Titanic)∧directed.in(e,1997)
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
17
Graph MatchingMap Ungrounded Graph to Knowledge Graph
Titanic
e
Cameron directed e
e
1997
direct
ed
.arg1
direct
ed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
directed.arg1(e, Cameron) ∧directed.arg2(e, Titanic) ∧ directed.in(e, 1997)
Titanic
m
Cameron directed n
1997
film.di
rected
by
.arg2
film.di
rected
by
.arg1
film
.initia
lrelea
sedate.arg
2film
.initia
lrelea
sedate.arg
1
film.directed by.arg2(m, Cameron) ∧film.directed by.arg1(m, Titanic) ∧
film.initial release date.arg1(n, Titanic) ∧film.initial release date.arg2(n, 1997)
Siva Reddy (University of Edinburgh)Large-scale Semantic Parsing without Question-Answer Pairs School of Informatics, University of Edinburgh 15
Ungrounded Graph Grounded Graph
18
Graph MatchingMap Ungrounded Graph to Knowledge Graph
Titanic
e
Cameron directed e
e
1997
direct
ed
.arg1
direct
ed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
directed.arg1(e, Cameron) ∧directed.arg2(e, Titanic) ∧ directed.in(e, 1997)
Titanic
m
Cameron directed n
1997
film.di
rected
by
.arg2
film.di
rected
by
.arg1
film
.initia
lrelea
sedate.arg
2film
.initia
lrelea
sedate.arg
1
film.directed by.arg2(m, Cameron) ∧film.directed by.arg1(m, Titanic) ∧
film.initial release date.arg1(n, Titanic) ∧film.initial release date.arg2(n, 1997)
Siva Reddy (University of Edinburgh)Large-scale Semantic Parsing without Question-Answer Pairs School of Informatics, University of Edinburgh 15
Ungrounded Graph Grounded Graph
18
Graph MatchingMap Ungrounded Graph to Knowledge Graph
Titanic
e
Cameron directed e
e
1997
direct
ed
.arg1
direct
ed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
directed.arg1(e, Cameron) ∧directed.arg2(e, Titanic) ∧ directed.in(e, 1997)
Titanic
m
Cameron directed n
1997
film.di
rected
by
.arg2
film.di
rected
by
.arg1
film
.initia
lrelea
sedate.arg
2film
.initia
lrelea
sedate.arg
1
film.directed by.arg2(m, Cameron) ∧film.directed by.arg1(m, Titanic) ∧
film.initial release date.arg1(n, Titanic) ∧film.initial release date.arg2(n, 1997)
Siva Reddy (University of Edinburgh)Large-scale Semantic Parsing without Question-Answer Pairs School of Informatics, University of Edinburgh 15
Ungrounded Graph Grounded Graph
18
Graph Mismatch
What is the name of the director of Titanic?
name target director
x e1 y e2 Titanicname.arg0
name.of director.arg0
director.of
type
type
Ungrounded graph
film.director
target
x m Titanicfilm.directed by
.arg2film.directed by
.arg1
type
Grounded graph
19
Graph Mismatch
What is the name of the director of Titanic?
name target director
x e1 y e2 Titanicname.arg0
name.of director.arg0
director.of
type
type??
Ungrounded graph
film.director
target
x m Titanicfilm.directed by
.arg2film.directed by
.arg1
type
Grounded graph
19
Graph Mismatch
What is the name of the director of Titanic?
name target director
x e1 y e2 Titanicname.arg0
name.of director.arg0
director.of
type
typeCONTRACT
Ungrounded graph
film.director
target
x m Titanicfilm.directed by
.arg2film.directed by
.arg1
type
Grounded graph
19
Graph Mismatch
What is the name of the director of Titanic?
name target director
y e2 Titanicdirector.arg0 director.of
type type
Ungrounded graph
film.director
target
x m Titanicfilm.directed by
.arg2film.directed by
.arg1
type
Grounded graph
19
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Feature Function
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Grounded Graph
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Ungrounded Graph
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Question
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Weight Vector
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
Learning Model
Structured Perceptron: Ranks grounded and ungroundedgraph pairs
(g, u) = argmaxg,u
Φ(g,u,q,KB) ·θ
39 / 64
Weight Vector
Training: Use gold graph to update weights
θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)
20
Our Approach: Recap
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
21
Syntax Options
CCG [Steedman, 2000; Bos et al., 2004]
HPSG [Copestake et al., 2001]
LFG [Dalrymple et al., 1995]
TAG [Joshi et al., 1995]
Dependency Trees [Tesniere, 1959]]
22
Universal Dependencies
Common syntactic representation across languages
Treebanks in 51 languages
40 dependency labels
22
Why from dependencies?Adopted from Manning’s laws
Easy to annotate
Treebanks in many languages
Very accurate parsers[Andor et al., 2016, Dyer et al., 2015, Chen & Manning, 2014]
Friendly to read
23
Dependency Tree to Semantics
Dependencies lack a formal theory of semantics
24
Dependency Tree to Semantics
Principle of Compositionality: the semantics of a complexexpression is determined by the semantics of its constituentexpressions and the rules used to combine them
Complex expression is the dependency tree
Constituent expressions are subtrees
Rules are the dependency labels
25
Dependency Tree to Semantics
Principle of Compositionality: the semantics of a complexexpression is determined by the semantics of its constituentexpressions and the rules used to combine them
Complex expression is the dependency tree
Constituent expressions are subtrees
Rules are the dependency labels
25
Dependency Tree to Semantics
Principle of Compositionality: the semantics of a complexexpression is determined by the semantics of its constituentexpressions and the rules used to combine them
Complex expression is the dependency tree
Constituent expressions are subtrees
Rules are the dependency labels
25
Dependency Tree to Semantics
Principle of Compositionality: the semantics of a complexexpression is determined by the semantics of its constituentexpressions and the rules used to combine them
Complex expression is the dependency tree
Constituent expressions are subtrees
Rules are the dependency labels
25
Universal Semantic Parsing
1. Dependencies to logical forms
2. Compositional
3. Language-agnostic conversion
I No token-specific semantics
I Dependency labels and postags dictate thesemantics
26
Compositional
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
27
Compositional
Disney acquired Pixar
nsubj dobj
root
Dependency labels drive the composition
28
Compositional
Disney acquired Pixar
nsubj dobj
root
. . . > dobj > · · ·> nsubj > .. .
(dobj acquired Pixar)
28
Compositional
Disney acquired Pixar
nsubj dobj
root
. . . > dobj > · · ·> nsubj > .. .
(dobj acquired Pixar)
28
Compositional
Disney acquired Pixar
nsubj dobj
root
. . . > dobj > · · ·> nsubj > .. .
(dobj acquired Pixar)
28
Compositional
Disney acquired Pixar
nsubj dobj
root
. . . > dobj > · · ·> nsubj > .. .
(nsubj (dobj acquired Pixar) Disney)
28
Compositional
Disney acquired Pixar
nsubj dobj
root
. . . > dobj > · · ·> nsubj > .. .
(nsubj (dobj acquired Pixar) Disney)
28
Compositional
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)(nsubj (dobj acquired Pixar) Disney)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
28
Language-agnostic Conversion
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for words
VERB⇒ λx.word(xe),e.g.,acquired⇒ λx.acquired(xe)
PROPN⇒ λx.word(xa),e.g.,Pixar⇒ λx.Pixar(xa)
29
Language-agnostic Conversion
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for dependency labels
dobj⇒ λ f λg λz . ∃x . f(z) ∧ g(x) ∧ arg2(ze,xa)
30
Language-agnostic ConversionDependencies to Logical FormsLambda Calculus
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for dependency labels
dobj⇒ λ f λg λz . ∃x . f(z) ∧ g(x) ∧ arg2(ze,xa)
This operation mirrors the tree structure
1831
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λfλgλz.∃y. λz.acquired(ze) λy.Pixar(ya) λx.Disney(xa)f(z) ∧ g(x)∧ f(z) ∧ g(y)∧arg1(ze, xa) arg2(ze, ya)
λgλz. ∃y. acquired(ze) ∧ g(y)∧ arg2(ze, ya)
λz. ∃y. acquired(ze) ∧ Pixar(ya)∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
32
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λfλgλz.∃y. λz.acquired(ze) λy.Pixar(ya) λx.Disney(xa)f(z) ∧ g(x)∧ f(z) ∧ g(y)∧arg1(ze, xa) arg2(ze, ya)
λgλz. ∃y. acquired(ze) ∧ g(y)∧ arg2(ze, ya)
λz. ∃y. acquired(ze) ∧ Pixar(ya)∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
32
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λfλgλz.∃y. λz.acquired(ze) λy.Pixar(ya) λx.Disney(xa)f(z) ∧ g(x)∧ f(z) ∧ g(y)∧arg1(ze, xa) arg2(ze, ya)
λgλz. ∃y. acquired(ze) ∧ g(y)∧ arg2(ze, ya)
λz. ∃y. acquired(ze) ∧ Pixar(ya)∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
32
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λx.Disney(xa)f(z) ∧ g(x)∧ λz. ∃y. acquired(ze) ∧ Pixar(ya)arg1(ze, xa) ∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
33
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λx.Disney(xa)f(z) ∧ g(x)∧ λz. ∃y. acquired(ze) ∧ Pixar(ya)arg1(ze, xa) ∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
33
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λx.Disney(xa)f(z) ∧ g(x)∧ λz. ∃y. acquired(ze) ∧ Pixar(ya)arg1(ze, xa) ∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
33
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λx.Disney(xa)f(z) ∧ g(x)∧ λz. ∃y. acquired(ze) ∧ Pixar(ya)arg1(ze, xa) ∧ arg2(ze, ya)
λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
33
Dependencies to Logical FormsComposition
Disney acquired Pixar
nsubj dobj
root
(nsubj (dobj acquired Pixar) Disney)
λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)
1
33
Dependencies to Logical Forms
Disney the company
appos
appos =λ f λgλx. f (x)∧g(x)
running horseVERB NOUN
amod amod =λ f λgλx.∃z. f (x)∧g(z)∧
amodi(ze,xa)
34
Dependencies to Logical Forms
Pixar is a company located in CA
nsubjcop
det aclcase
nmod
root
35
Dependencies to Logical Forms
Pixar is a company located in CA
nsubjcop
det aclcase
nmod
arg2
argin
root
λx.∃yz. located(ze)∧Pixar(xa)∧CA(ya)∧company(xa)∧ arg2(ze,xa)∧ argin(ze,ya)
35
UDepLambda
Pixar is a company located in CA
nsubjcop
det aclcase
nmod
root
. . . > dobj > · · ·> nsubj > .. .
... (acl company (nmod located (case CA in) ))...λfgx.∃z. λx.compay(xa) λfgz.∃x. λx.located(xe) λfgx.f(x) λx.CA(xa) λx.empty(x)
f(x) ∧ g(z)∧ f(z) ∧ g(x)arg2(ze, xa) argin(ze, xa) λx. CA(xa)
1
lambda expression composition
∃z.company(Pixar)∧ located(ze)∧ arg2(ze,Pixar)∧ argin(ze,CA)
36
UDepLambda in a nutshell
Dependency tree is a series of compositions
Dependency label defines the composition function
Each function takes two typed-semantic sub-expressions
Returns typed-semantics of the larger expression
37
Limitations
Context-sensitive semantics of dependency labelsI John broke the windowI The window broke
Delexicalized context is not sufficiente.g., quantifiers vs determiners
38
Quantifiers and Negation Scope
Higher-order type system
Fine-grained dependency labels
Fancellu, Reddy, Lopez, Webber (UD Workshop 2017)Supplementary of Reddy et al. (EMNLP 2017)
39
Experimental Setup
69 lambda calculus formulae
BiLSTM Dependency Parser [Kipperwiser and Goldberg (2016)]
I English: 81.8I German: 74.7I Spanish: 82.2
40
Baselines
SIMPLEGRAPH: All entities connected to a single event(Syntax agnostic but uses domain knowledge)
DEPTREE: Transduce a dependency tree to target graph(Syntax to Machine Representation directly)
41
Results on Multilingual GraphQuestions
en NASA has how many launch sites?de Wie viele Abschussbasen besitzt NASA?es ¿Cuantos sitios de despegue tiene NASA?
en Which loudspeakers are heavier than 82.0 kg?de Welche Lautsprecher sind schwerer als 82.0 kg?es ¿Que altavoces pesan mas de 82.0 kg?
42
Results on Multilingual GraphQuestionsEnglish
Sim
pleGraph
DepTree
UDepLambda
ParaSempre
Seq2Graph
Induced
12.5
13
13.5
14
14.5
15
15.5
16
16.5
17
17.5
18
15.9 16
17.7
12.8
16.3
17
15.9 16
17.7
17
Average
F1
1
43
Results on Multilingual GraphQuestionsSpanish
Sim
pleGraph
DepTree
UDepLambda
11.2
11.4
11.6
11.8
12
12.2
12.4
12.6
12.8
11.411.3
12.8
11.411.3
12.8
Average
F1
1
44
Results on Multilingual GraphQuestionsGerman
Sim
pleGraph
DepTree
UDepLambda
8.2
8.4
8.6
8.8
9
9.2
9.4
9.6
8.8
8.3
9.5
8.8
8.3
9.5
Average
F1
1
45
Part-1 Summary
Treebank Syntax helps Semantic Parsing
Converting Syntax to Logic to Machine Representation isbetter than direct conversion
Dependencies to Logic is promising(e.g., can handle control, quantification and negation scope)
46
Semantic Parsing with Machine-induced NL Structures
Cheng, Reddy, Saraswat, Lapata (ACL 2017)
47
Motivation
NaturalLanguage
Domain-specificNL Structure
MachineRepresentation
MachineInduced
1. Useful representations for the task at hand
2. Interpretablity and trouble-shooting
48
Learning NL Induced Structures
NaturalLanguage
Domain-specificNL Structure
MachineRepresentationp(u|x) – Transition-based Stack-LSTM
(Dyer et al. 2016)p(g|u)
Input: Pixar is a company located in CA
Actions Stack StateNon Terminal Shift located(Non Terminal Shift located(company(Terminal Shift located(company(PixarReduce located(company(Pixar)Terminal Shift located(company(Pixar), CAReduce located(company(Pixar), CA)
49
Learning NL Induced Structures
NaturalLanguage
Domain-specificNL Structure
MachineRepresentationp(u|x) p(g|u) – Bilinear Model
Input: Pixar is a company located in CA
Ungrounded: located(company(Pixar), CA)
Grounded: headquarters(organization(Pixar), CA)
50
Induced Structures to Machine Representation
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
51
Induced Structures to Machine Representation
Sentence Syntax UngroundedLogical Form
Titanic
e
Cameron e
e
1997
directed
.arg1
directed
.arg2
directed
.indirected
.arg2
directed.arg1
directed.in
Ungrounded Graph
KB
Titanic
m
Cameron n
1997
film.d
irect
edby
.arg2
film.d
irect
edby
.arg1
film
.initia
lrelea
sed
ate.arg
2fi
lm.in
itial
release
da
te.arg1
Grounded Graph
Denotation-basedLearning ModelWho is the director of Titanic? James Cameron
SyntacticParser Syntax to Logic
Graph Matching
Machine-induced NL Representation
51
Results on GraphQuestionsEnglish
Sim
pleGraph
DepTree
UDepLambda
ParaSempre
Seq2Graph
Induced
12.5
13
13.5
14
14.5
15
15.5
16
16.5
17
17.5
18
15.9 16
17.7
12.8
16.3
17
15.9 16
17.7
17Average
F1
1
52
Results on Spades∗ (79k train, 5k test)
Seq2Graph
Linguistic
Induced
28.5
29
29.5
30
30.5
31
31.5
28.6
30.9
31.5
28.6
30.9
31.5
Average
F1
1
*Bisk, Reddy, Blitzer, Hockenmaier, Steedman (EMNLP 2016) 53
Part-2 Summary
Induced structures start rivaling liguists structureswith increase in training data
54
Linguists vs MachinesPostive cases
Boeing was founded in 1916 and is headquartered inblank
blank has confirmed to play captain Haddock .
Mathematica is a product of blank .
blank is a corporation that is owned by the Edmonton city
55
Linguists vs MachinesPostive cases
Boeing was founded in 1916 and is headquartered inblank
blank has confirmed to play captain Haddock .
Mathematica is a product of blank .
blank is a corporation that is owned by the Edmonton city
55
Linguists vs MachinesNegative cases
Which university did Obama go to?
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
Romney is the worst governor that has blank ever had.
56
Linguists vs MachinesNegative cases
Which university did Obama go to?
I university(x,Obama) = education at(x,Obama)
Eddie dumped Debbie to marry blank .
Wilhelm Maybach and his son blank started Maybach.
Romney is the worst governor that has blank ever had.
56
Future Direction
Graph Convolutional Networks for Graph Matching
Rich type-system for Dependencies
Linguistic knowledge transfer for Deep Learning
57
Summary
Natural Language Structure helps Semantic Parsing
But what is the best structure?
Code
I Dependency Experiments:https://github.com/sivareddyg/UDepLambda
Demo: https://sivareddy.in/udeplambda.html
I Neural Experiments:https://github.com/cheng6076/scanner
Thank You!58
Results on WebQuestions
SimpleG
raph
DepTree
UDepLambda
UDepLambdaSRL
48.4
48.6
48.8
49
49.2
49.4
49.6
49.8
48.5
48.8
49.5
49.8
48.5
48.8
49.5
49.8
Average
F1
1
59
Results on WebQuestions
Sim
pleGraph
CCGGraph
DepTree
UDepLambda
DepLambda
Yih15
Xu16
48.5
49
49.5
50
50.5
51
51.5
52
52.5
53
53.5
48.5 48.648.8
49.5
50.3
52.5
53.3
48.5 48.648.8
49.5
50.3
Average
F1
1
60
Learning Model
? We do not have access to gold graphs
? Access only to the answers rather than the query
? Solution: use a surrogate gold graph
Surrogate Gold Graph:
Learning Model
We do not have access to gold graphs
Access only to the answers rather than the query
Instead use a surrogate gold graph
Surrogate Gold Graph:
(g+,u+) = argmax(g,u)∈O(q)
Φ(g,u,q,KB) ·θ t
40 / 64
Oracle Graphs
61
Learning Model
? We do not have access to gold graphs
? Access only to the answers rather than the query
? Solution: use a surrogate gold graph
Surrogate Gold Graph:
Learning Model
We do not have access to gold graphs
Access only to the answers rather than the query
Instead use a surrogate gold graph
Surrogate Gold Graph:
(g+,u+) = argmax(g,u)∈O(q)
Φ(g,u,q,KB) ·θ t
40 / 64
Oracle Graphs
61
Dependencies to Logical FormsSingle Type System
Disney acquired Pixar
nsubj dobj
root
All constituents are of the same lambda expression typeTYPE[acquired] = TYPE[Pixar] = TYPE[(dobj acquired Pixar)]
61
Dependencies to Logical FormsSingle Type System
Disney acquired Pixar
nsubj dobj
root
All words have a lambda expression of type ηI TYPE[acquired] = η
I TYPE[Pixar] = η
I TYPE[(dobj acquired Pixar)] = η
=⇒ TYPE[dobj] = η → η → η
61
Dependencies to Logical FormsSingle Type System
Disney acquired Pixar
nsubj dobj
root
All constituents have a lambda expression of type ηI TYPE[acquired] = η
I TYPE[Pixar] = η
I TYPE[(dobj acquired Pixar)] = η
=⇒ TYPE[dobj] = η → η → η
61
Dependencies to Logical FormsSingle Type System
Disney acquired Pixar
nsubj dobj
root
All constituents have a lambda expression of type ηI TYPE[acquired] = η
I TYPE[Pixar] = η
I TYPE[(dobj acquired Pixar)] = η
=⇒ TYPE[dobj] = η → η → η61
Dependencies to Logical FormsLambda Calculus for Single Type System
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for words
acquired⇒ λxe.acquired(xe)
⇒TYPE = Event→ Bool
Pixar⇒ λxa.Pixar(xa)
⇒TYPE = Ind→ Bool
Here TYPE[acquired] 6= TYPE[Pixar] 7
61
Dependencies to Logical FormsLambda Calculus for Single Type System
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for words
acquired⇒ λxe.acquired(xe) ⇒TYPE = Event→ BoolPixar⇒ λxa.Pixar(xa) ⇒TYPE = Ind→ Bool
Here TYPE[acquired] 6= TYPE[Pixar] 7
61
Dependencies to Logical FormsLambda Calculus
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for dependency labels
dobj⇒ λ f λg λz . ∃x . f(z) ∧ g(x) ∧ arg2(ze,xa)
This operation mirrors the tree structure
62
Dependencies to Logical FormsLambda Calculus
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for dependency labels
dobj⇒ λ f λg λz . ∃x . f(z) ∧ g(x) ∧ arg2(ze,xa)
This operation mirrors the tree structure
62
Dependencies to Logical FormsLambda Calculus for Single Type System
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for words
acquired⇒ λxaxe.acquired(xe)
⇒TYPE = Ind×Event→ Bool
Pixar⇒ λxaxe.Pixar(xa)
⇒TYPE = Ind×Event→ Bool
Here η = TYPE[acquired] = TYPE[Pixar] X
62
Dependencies to Logical FormsLambda Calculus for Single Type System
Disney acquired Pixar
nsubj dobj
root
Lambda Expression for words
acquired⇒ λxaxe.acquired(xe) ⇒TYPE = Ind×Event→ BoolPixar⇒ λxaxe.Pixar(xa) ⇒TYPE = Ind×Event→ Bool
Here η = TYPE[acquired] = TYPE[Pixar] X
62
ConjunctionsSentence:
Eminem signed to Interscope and discovered 50 Cent.
Binarized tree:(nsubj (conj-vp (cc s to I and) d 50) Eminem)
Substitution:
conj-vp⇒ λ fgx.∃yz. f (y)∧g(z)∧ coord(x,y,z)
Logical Expression:
λw.∃xyz.Eminem(xa)∧ coord(w,y,z)∧arg1(we,xa)∧ s to I(y)∧d 50(z)
Post processing:
λe.∃xyz.Eminem(xa)∧ arg1(ye,xa)∧arg1(ze,xa)∧ s to I(y)∧d 50(z)
62
ConjunctionsSentence:
Eminem signed to Interscope and discovered 50 Cent.
Binarized tree:(nsubj (conj-vp (cc s to I and) d 50) Eminem)
Substitution:
conj-vp⇒ λ fgx.∃yz. f (y)∧g(z)∧ coord(x,y,z)
Logical Expression:
λw.∃xyz.Eminem(xa)∧ coord(w,y,z)∧arg1(we,xa)∧ s to I(y)∧d 50(z)
Post processing:
λe.∃xyz.Eminem(xa)∧ arg1(ye,xa)∧arg1(ze,xa)∧ s to I(y)∧d 50(z)
62
ConjunctionsSentence:
Eminem signed to Interscope and discovered 50 Cent.
Binarized tree:(nsubj (conj-vp (cc s to I and) d 50) Eminem)
Substitution:
conj-vp⇒ λ fgx.∃yz. f (y)∧g(z)∧ coord(x,y,z)
Logical Expression:
λw.∃xyz.Eminem(xa)∧ coord(w,y,z)∧arg1(we,xa)∧ s to I(y)∧d 50(z)
Post processing:
λe.∃xyz.Eminem(xa)∧ arg1(ye,xa)∧arg1(ze,xa)∧ s to I(y)∧d 50(z)
62
Graph Transformation: CONTRACT operation
What language do the people in Ghana speak?
language target people
x e1 y e2 Ghanaspeak.arg2
speak.arg1
people.arg1
people.nmod.in
type
type
Ungrounded graph
language.human language
target
x m Ghanalocation.country
.official language.2location.country
.official language.1
type
Grounded graph
62
Graph Mismatch: EXPAND operation
What to do Washington DC December?
Before EXPAND
I λ z.∃xyw. TARGET(xa)∧do(ze)∧ arg1(ze,xa)∧Washington DC(ya)∧December(wa)
After EXPAND
I λ z.∃xyw. TARGET(xa)∧do(ze)∧ arg1(ze,xa)∧Washington DC(ya)∧dep(ze,ya)∧December(wa)∧dep(ze,wa)
62
Dependency Graphs to Logical Forms
Anna wants to marry Kristoff
nsubj
xcomp
mark dobj
nsubj
Anna wants to marry Kristoff
Ω Ω
nsubj
xcomp
mark dobj
bind nsubj
63
Dependency Graphs to Logical Forms
Anna wants to marry Kristoff
Ω Ω
nsubj
xcomp
mark dobj
bind nsubj
Substitution ExpressionsBIND = λ f λgλx. f (x)∧g(x)xcomp = λ fgx.∃y. f (x)∧g(y)∧xcomp(xe,ye)ω = λx.EQ(x,ω)
Final Expression:
λ z.∃xyw.wants(ze)∧Anna(xa)∧ arg1(ze,xa)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ Kristoff(wa)∧ arg2(ye,wa) .
64