Linguists-defined vs. Machine-induced Natural Language ... · Tom Kwiatkowski Michael Collins Slav...

Linguists-defined vs. Machine-inducedNatural Language Structures for

Executable Semantic Parsing

Siva ReddyStanford NLP Group

1

Mirella Lapata Mark Steedman Oscar Tackstrom

Tom Kwiatkowski Michael Collins Slav Petrov

Dipanjan Das Jianpeng Cheng Vijay Saraswat

2

Semantic Parsing

Natural Language (NL)

Machine Executable Representation (MR)P

arser

3

Semantic Parsing

NL

Leonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.

MR cast(TITANIC,DICAPRIO,JACK)∧

director(TITANIC,CAMERON)

TRUE

Parser

KB

Execute

7.7/10 · IMDb

88% · Rotten Tomatoes

James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More

Initial release: November 18, 1997 (London)

Director: James Cameron

Featured song: My Heart Will Go On

Cast

1997 ‧ Drama film/Romance ‧ 3h 30m

Titanic

LeonardoDiCaprioJack Dawson

KateWinsletRose DeWittBukater

Billy ZaneCaledonHockley

GloriaStuartRose DeWittBukater

Kathy BatesMolly Brown

Feedback

Edinburgh - From your Internet address - Use precise location - Learn more

Help Send feedback Privacy Terms

titanic 1997 Sign in

titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1

1 of 1 29/05/15 20:39

4

Semantic ParsingN

LLeonardo DiCaprio starredas Jack in Titanic whichwas directed by JamesCameron.

MR cast(TITANIC,DICAPRIO,JACK)∧

director(TITANIC,CAMERON)

TRUE

Parser

KB

Execute

7.7/10 · IMDb

88% · Rotten Tomatoes

James Cameron's "Titanic" is an epic, action-packed romance set againstthe ill-fated maiden voyage of the R.M.S. Titanic; the pride and joy of theWhite Star Line and, at the time, the larg… More

Initial release: November 18, 1997 (London)

Director: James Cameron

Featured song: My Heart Will Go On

Cast

1997 ‧ Drama film/Romance ‧ 3h 30m

Titanic

LeonardoDiCaprioJack Dawson

KateWinsletRose DeWittBukater

Billy ZaneCaledonHockley

GloriaStuartRose DeWittBukater

Kathy BatesMolly Brown

Feedback

Edinburgh - From your Internet address - Use precise location - Learn more

Help Send feedback Privacy Terms

titanic 1997 Sign in

titanic 1997 - Google Search https://www.google.co.uk/search?sclient=psy-ab&espv=2&biw=2133&bih=1052&btnG=Search&q=titanic+1997&oq=&gs_l=&pbx=1

1 of 1 29/05/15 20:39

4

Application: Question Answering

Feedback

James CameronTitanic, Director

James Francis Cameron is a Canadian filmmaker, inventor, engineer, philanthropist, anddeep-sea explorer who has directed the two biggest box office films of all time. He firstfound major success with the science-fiction hit The Terminator. Wikipedia

Movies and overview

James Cameron - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/James_CameronHe then became a popular Hollywood director and was hired to write and direct ..... in

the TV documentary special Titanic: The Final Word with James Cameron, ...James Cameron filmography - Suzy Amis - The Abyss - Gale Anne Hurd

Titanic (1997 film) - Wikipedia, the free encyclopediaen.wikipedia.org/wiki/Titanic_(1997_film)Titanic is a 1997 American epic romantic disaster film directed, written, co-produced, and

co-edited by James Cameron.Gloria Stuart - Heart of the Ocean - Danny Nucci - J. Bruce Ismay

James Cameron - IMDbwww.imdb.com/name/nm0000116/James Cameron, Writer: Aliens. ... He landed his first professional film job as art

director, miniature-set builder, and process-projection ... 1997 Titanic (written by).

James Cameron - Biography - Director, Producer ...www.biography.com/people/james-cameron-54657017 Mar 2015 - Oscar-winning director James Cameron is best known for the highly

acclaimed, box-office hits Aliens (1986), Titanic (1997) and Avatar (2009).

'Titanic' director denies Jack and Rose could have survived ...www.nme.com/filmandtv/news/-titanic-director-denies...-/28658614 Sep 2012 - Titanic director James Cameron has dismissed an internet rumour

claiming that the movie's doomed lovers Jack and Rose could both have ...

Biography: Titanic and Avatar Director James Cameron ...www.watchmojo.com › Videos › Film › MoviesBiography: Titanic and Avatar Director James Cameron He's the creative genius

behind cinema's most iconic films, such as The Terminator, The Abyss and ...

Who was the director of Titanic? - Quorawww.quora.com › Movie Business and Industry › Movie ProductionTitanic (1997 movie): Why didn't Jack climb onto the plank with Rose when there ...

Titanic (1997 movie): James Cameron more or less copied the line "he exists ...

'Titanic' Director James Cameron Speaks about Lasting ...www.spiegel.de › English Site › Zeitgeist › History6 Apr 2012 - Canadian film director James Cameron has made three films about the

Titantic , including the 1997 blockbuster of the same name.

Titanic - Cast, Crew, Director and Awards - NYTimes.comwww.nytimes.com/movies/movie/158894/Titanic/detailsFull Acting Credits for Titanic » ... Director - James Cameron ... Best Director - James

[di/wr] Cameron - 1997 Broadcast Film Critics Association; Best Director ...

'Titanic' and 'Avatar' director James Cameron reaches ocean'slatimesblogs.latimes.com/.../titanic-and-avatar-director-james-cameron-re...

Freebase Question Answering

who is the director of titanic Sign in

who is the director of titanic - Google Search https://www.google.co.uk/search?client=ubuntu&channel=fs&q=who+is+the+director+of+titanic&ie=utf-8&oe=utf-8&gfe_rd=cr&ei=gtNoVd71M...

1 of 2 29/05/15 22:25

5

Other Semantic Parsing Applications

6

This Talk

NaturalLanguage

MachineRepresentation

vs.

NaturalLanguage Syntax Machine

Representation

SyntacticParser

vs.

NaturalLanguage

Domain-specificNL Structure


MachineInduced

7

Outline

1. Semantic Parsing with Linguists-defined NL Structures

2. Semantic Parsing with Machine-induced NL Structures

8

Semantic Parsing with Linguists-defined NL Structures

Reddy, Lapata, Steedman (TACL 2014)Reddy, Tackstrom, Collins, Kwiatkowski, Das, Steedman, Lapata (TACL 2016)Reddy, Tackstrom, Petrov, Steedman, Lapata (EMNLP 2017)

9

Motivation


Representation

SyntacticParser

1. Decades of research on syntax and parsing

2. Syntax is task-agnostic and universal

3. Annotation does not require target-domain experts

10

Motivation


Representation

SyntacticParser

1. Semantic Parsing data is expensive

2. Many treebanks

3. Constrained search

11

End-to-End Semantic Parsing[Zelle & Mooney, 1996; Zettlemoyer & Collins, 2005; Kwiatkowski et al., 2010; Liang et al., 2011;Artzi & Zettlemoyer, 2011; Krishnamurthy & Mitchell, 2012; Berant et al., 2013; Pasupat & Liang, 2015;

Yih et al., 2015]

Grammar learning problem

I director→ N : λx.film.director(x)

I of→ (NP\NP)/NP :λ f λgλx.∃y∃e. f (y)∧g(x)∧film.directed by(e)∧arg1(e,y)∧arg2(e,x)

Question

Who is the director of Titanic?

Grounded Logical Form

λx.∃e. film.director(x) ∧film.directed by(e) ∧

arg1(e,Titanic)∧ arg2(e,x)

12

Intermediate Semantic Parsing[Kwiatkowski et al., 2013; Reddy et al., 2014; Choi et al., 2015; Artzi et al., 2015]

Language toungrounded logical form

Ungrounded logical form togrounded logical form

Question

Who is the director of Titanic?

Ungrounded Logical Form

λx.TARGET(xa)∧ director(xa)∧director event(xe) ∧

arg0(xe,xa)∧ arg.of(xe,Titanic)

Grounded Logical Form

λx.∃e. film.director(x) ∧film.directed by(e) ∧

arg2(e,x)∧ arg1(e,Titanic)

13

Task Setting

Training Data: Question and Answer Pairs

Evaluation: Question Answering on Freebase

Resources: Syntactic Parser, Syntax to Ungrounded Logic

14

Freebase is a Graphusa

natashaobama

p q USpresident

r s

ColumbiaUniversity

mBarackObama

nMichelleObama

m m n n

education.university

Bachelorof Arts

1992

headquarters

.country

headquarters

.organisation

person.

nationality.arg2

person.

nationality.arg1

person.

parents.arg2

person.

parents.arg1

perso

n.

paren

ts.arg2

perso

n.

paren

ts.arg1

education.institution

education.student

marriage.spouse

marriage.spouse

education

.institution

education.degree ed

ucation

.deg

ree

education

.studen

t

marriage

.from

marriage

.spouse

marriage

.from

marriage

.spouse

type

type

15

Our Approach

Sentence Syntax UngroundedLogical Form

Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph

Denotation-basedLearning ModelWho is the director of Titanic? James Cameron

SyntacticParser Syntax to Logic

Graph Matching

16

Our Approach


Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph

Denotation-basedLearning Model


Graph Matching

16

Our Approach


Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph



Graph Matching

16

Logical Form to Ungrounded GraphCameron directed Titanic in 1997

λe.directed.arg1(e,Cameron)∧directed.arg2(e,Titanic)∧directed.in(e,1997)

Titanic

Cameron

1997

17



Titanic

e

Cameron e

e

1997

17



Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

17

Graph MatchingMap Ungrounded Graph to Knowledge Graph

Titanic

e

Cameron directed e

e

1997

direct

ed

.arg1

direct

ed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

directed.arg1(e, Cameron) ∧directed.arg2(e, Titanic) ∧ directed.in(e, 1997)

Titanic

m

Cameron directed n

1997

film.di

rected

by

.arg2

film.di

rected

by

.arg1

film

.initia

lrelea

sedate.arg

2film

.initia

lrelea

sedate.arg

1

film.directed by.arg2(m, Cameron) ∧film.directed by.arg1(m, Titanic) ∧

film.initial release date.arg1(n, Titanic) ∧film.initial release date.arg2(n, 1997)

Siva Reddy (University of Edinburgh)Large-scale Semantic Parsing without Question-Answer Pairs School of Informatics, University of Edinburgh 15

Ungrounded Graph Grounded Graph

18

Graph Mismatch

What is the name of the director of Titanic?

name target director

x e1 y e2 Titanicname.arg0

name.of director.arg0

director.of

type

type

Ungrounded graph

film.director

target

x m Titanicfilm.directed by

.arg2film.directed by

.arg1

type

Grounded graph

19

Graph Mismatch





director.of

type

type??

Ungrounded graph

film.director

target



.arg1

type

Grounded graph

19

Graph Mismatch





director.of

type

typeCONTRACT

Ungrounded graph

film.director

target



.arg1

type

Grounded graph

19

Graph Mismatch



y e2 Titanicdirector.arg0 director.of

type type

Ungrounded graph

film.director

target



.arg1

type

Grounded graph

19

Learning Model

Structured Perceptron: Ranks grounded and ungroundedgraph pairs

Learning Model


(g, u) = argmaxg,u

Φ(g,u,q,KB) ·θ

39 / 64

Feature Function

Training: Use gold graph to update weights

θ ← θ +Φ(g+,u+,q,KB)−Φ(g, u,q,KB)

20

Learning Model


Learning Model


(g, u) = argmaxg,u

Φ(g,u,q,KB) ·θ

39 / 64

Grounded Graph



20

Learning Model


Learning Model


(g, u) = argmaxg,u

Φ(g,u,q,KB) ·θ

39 / 64

Ungrounded Graph



20

Learning Model


Learning Model


(g, u) = argmaxg,u

Φ(g,u,q,KB) ·θ

39 / 64

Question



20

Learning Model


Learning Model


(g, u) = argmaxg,u

Φ(g,u,q,KB) ·θ

39 / 64

Weight Vector



20

Our Approach: Recap


Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph



Graph Matching

21

Syntax Options

CCG [Steedman, 2000; Bos et al., 2004]

HPSG [Copestake et al., 2001]

LFG [Dalrymple et al., 1995]

TAG [Joshi et al., 1995]

Dependency Trees [Tesniere, 1959]]

22

Universal Dependencies

Common syntactic representation across languages

Treebanks in 51 languages

40 dependency labels

22

Why from dependencies?Adopted from Manning’s laws

Easy to annotate

Treebanks in many languages

Very accurate parsers[Andor et al., 2016, Dyer et al., 2015, Chen & Manning, 2014]

Friendly to read

23

Dependency Tree to Semantics

Dependencies lack a formal theory of semantics

24

Dependency Tree to Semantics

Principle of Compositionality: the semantics of a complexexpression is determined by the semantics of its constituentexpressions and the rules used to combine them

Complex expression is the dependency tree

Constituent expressions are subtrees

Rules are the dependency labels

25

Universal Semantic Parsing

1. Dependencies to logical forms

2. Compositional

3. Language-agnostic conversion

I No token-specific semantics

I Dependency labels and postags dictate thesemantics

26

Compositional

Disney acquired Pixar

nsubj dobj

root

(nsubj (dobj acquired Pixar) Disney)

λz.∃xy.acquired(ze) ∧ Pixar(ya) ∧Disney(xa)∧arg1(ze, xa) ∧ arg2(ze, ya)

1

27

Compositional


nsubj dobj

root

Dependency labels drive the composition

28

Compositional


nsubj dobj

root

. . . > dobj > · · ·> nsubj > .. .

(dobj acquired Pixar)

28

Compositional


nsubj dobj

root

. . . > dobj > · · ·> nsubj > .. .


28

Compositional


nsubj dobj

root

(nsubj (dobj acquired Pixar) Disney)(nsubj (dobj acquired Pixar) Disney)


1

28

Language-agnostic Conversion


nsubj dobj

root

Lambda Expression for words

VERB⇒ λx.word(xe),e.g.,acquired⇒ λx.acquired(xe)

PROPN⇒ λx.word(xa),e.g.,Pixar⇒ λx.Pixar(xa)

29

Language-agnostic Conversion


nsubj dobj

root

Lambda Expression for dependency labels

dobj⇒ λ f λg λz . ∃x . f(z) ∧ g(x) ∧ arg2(ze,xa)

30

Language-agnostic ConversionDependencies to Logical FormsLambda Calculus


nsubj dobj

root



This operation mirrors the tree structure

1831

Dependencies to Logical FormsComposition


nsubj dobj

root

(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λfλgλz.∃y. λz.acquired(ze) λy.Pixar(ya) λx.Disney(xa)f(z) ∧ g(x)∧ f(z) ∧ g(y)∧arg1(ze, xa) arg2(ze, ya)

λgλz. ∃y. acquired(ze) ∧ g(y)∧ arg2(ze, ya)

λz. ∃y. acquired(ze) ∧ Pixar(ya)∧ arg2(ze, ya)

λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)


1

32



nsubj dobj

root

(nsubj (dobj acquired Pixar) Disney)λfλgλz. ∃x. λx.Disney(xa)f(z) ∧ g(x)∧ λz. ∃y. acquired(ze) ∧ Pixar(ya)arg1(ze, xa) ∧ arg2(ze, ya)

λgλz.∃xy.acquired(ze) ∧ Pixar(ya) ∧ g(x)∧arg1(ze, xa) ∧ arg2(ze, ya)


1

33



nsubj dobj

root



1

33

Dependencies to Logical Forms

Disney the company

appos

appos =λ f λgλx. f (x)∧g(x)

running horseVERB NOUN

amod amod =λ f λgλx.∃z. f (x)∧g(z)∧

amodi(ze,xa)

34


Pixar is a company located in CA

nsubjcop

det aclcase

nmod

root

35



nsubjcop

det aclcase

nmod

arg2

argin

root

λx.∃yz. located(ze)∧Pixar(xa)∧CA(ya)∧company(xa)∧ arg2(ze,xa)∧ argin(ze,ya)

35

UDepLambda


nsubjcop

det aclcase

nmod

root

. . . > dobj > · · ·> nsubj > .. .

... (acl company (nmod located (case CA in) ))...λfgx.∃z. λx.compay(xa) λfgz.∃x. λx.located(xe) λfgx.f(x) λx.CA(xa) λx.empty(x)

f(x) ∧ g(z)∧ f(z) ∧ g(x)arg2(ze, xa) argin(ze, xa) λx. CA(xa)

1

lambda expression composition

∃z.company(Pixar)∧ located(ze)∧ arg2(ze,Pixar)∧ argin(ze,CA)

36

UDepLambda in a nutshell

Dependency tree is a series of compositions

Dependency label defines the composition function

Each function takes two typed-semantic sub-expressions

Returns typed-semantics of the larger expression

37

Limitations

Context-sensitive semantics of dependency labelsI John broke the windowI The window broke

Delexicalized context is not sufficiente.g., quantifiers vs determiners

38

Quantifiers and Negation Scope

Higher-order type system

Fine-grained dependency labels

Fancellu, Reddy, Lopez, Webber (UD Workshop 2017)Supplementary of Reddy et al. (EMNLP 2017)

39

Experimental Setup

69 lambda calculus formulae

BiLSTM Dependency Parser [Kipperwiser and Goldberg (2016)]

I English: 81.8I German: 74.7I Spanish: 82.2

40

Baselines

SIMPLEGRAPH: All entities connected to a single event(Syntax agnostic but uses domain knowledge)

DEPTREE: Transduce a dependency tree to target graph(Syntax to Machine Representation directly)

41

Results on Multilingual GraphQuestions

en NASA has how many launch sites?de Wie viele Abschussbasen besitzt NASA?es ¿Cuantos sitios de despegue tiene NASA?

en Which loudspeakers are heavier than 82.0 kg?de Welche Lautsprecher sind schwerer als 82.0 kg?es ¿Que altavoces pesan mas de 82.0 kg?

42

Results on Multilingual GraphQuestionsEnglish

Sim

pleGraph

DepTree

UDepLambda

ParaSempre

Seq2Graph

Induced

12.5

13

13.5

14

14.5

15

15.5

16

16.5

17

17.5

18

15.9 16

17.7

12.8

16.3

17

15.9 16

17.7

17

Average

F1

1

43

Results on Multilingual GraphQuestionsSpanish

Sim

pleGraph

DepTree

UDepLambda

11.2

11.4

11.6

11.8

12

12.2

12.4

12.6

12.8

11.411.3

12.8

11.411.3

12.8

Average

F1

1

44

Results on Multilingual GraphQuestionsGerman

Sim

pleGraph

DepTree

UDepLambda

8.2

8.4

8.6

8.8

9

9.2

9.4

9.6

8.8

8.3

9.5

8.8

8.3

9.5

Average

F1

1

45

Part-1 Summary

Treebank Syntax helps Semantic Parsing

Converting Syntax to Logic to Machine Representation isbetter than direct conversion

Dependencies to Logic is promising(e.g., can handle control, quantification and negation scope)

46

Semantic Parsing with Machine-induced NL Structures

Cheng, Reddy, Saraswat, Lapata (ACL 2017)

47

Motivation

NaturalLanguage



MachineInduced

1. Useful representations for the task at hand

2. Interpretablity and trouble-shooting

48

Learning NL Induced Structures

NaturalLanguage


MachineRepresentationp(u|x) – Transition-based Stack-LSTM

(Dyer et al. 2016)p(g|u)

Input: Pixar is a company located in CA

Actions Stack StateNon Terminal Shift located(Non Terminal Shift located(company(Terminal Shift located(company(PixarReduce located(company(Pixar)Terminal Shift located(company(Pixar), CAReduce located(company(Pixar), CA)

49

Learning NL Induced Structures

NaturalLanguage


MachineRepresentationp(u|x) p(g|u) – Bilinear Model

Input: Pixar is a company located in CA

Ungrounded: located(company(Pixar), CA)

Grounded: headquarters(organization(Pixar), CA)

50

Induced Structures to Machine Representation


Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph



Graph Matching

51

Induced Structures to Machine Representation


Titanic

e

Cameron e

e

1997

directed

.arg1

directed

.arg2

directed

.indirected

.arg2

directed.arg1

directed.in

Ungrounded Graph

KB

Titanic

m

Cameron n

1997

film.d

irect

edby

.arg2

film.d

irect

edby

.arg1

film

.initia

lrelea

sed

ate.arg

2fi

lm.in

itial

release

da

te.arg1

Grounded Graph



Graph Matching

Machine-induced NL Representation

51

Results on GraphQuestionsEnglish

Sim

pleGraph

DepTree

UDepLambda

ParaSempre

Seq2Graph

Induced

12.5

13

13.5

14

14.5

15

15.5

16

16.5

17

17.5

18

15.9 16

17.7

12.8

16.3

17

15.9 16

17.7

17Average

F1

1

52

Results on Spades∗ (79k train, 5k test)

Seq2Graph

Linguistic

Induced

28.5

29

29.5

30

30.5

31

31.5

28.6

30.9

31.5

28.6

30.9

31.5

Average

F1

1

*Bisk, Reddy, Blitzer, Hockenmaier, Steedman (EMNLP 2016) 53

Part-2 Summary

Induced structures start rivaling liguists structureswith increase in training data

54

Linguists vs MachinesPostive cases

Boeing was founded in 1916 and is headquartered inblank

blank has confirmed to play captain Haddock .

Mathematica is a product of blank .

blank is a corporation that is owned by the Edmonton city

55

Linguists vs MachinesNegative cases

Which university did Obama go to?


I university(x,Obama) = education at(x,Obama)

Eddie dumped Debbie to marry blank .

Wilhelm Maybach and his son blank started Maybach.

Romney is the worst governor that has blank ever had.

56







56

Future Direction

Graph Convolutional Networks for Graph Matching

Rich type-system for Dependencies

Linguistic knowledge transfer for Deep Learning

57

Summary

Natural Language Structure helps Semantic Parsing

But what is the best structure?

Code

I Dependency Experiments:https://github.com/sivareddyg/UDepLambda

Demo: https://sivareddy.in/udeplambda.html

I Neural Experiments:https://github.com/cheng6076/scanner

Thank You!58

https://github.com/sivareddyg/UDepLambda

https://sivareddy.in/udeplambda.html

https://github.com/cheng6076/scanner

Results on WebQuestions

SimpleG

raph

DepTree

UDepLambda

UDepLambdaSRL

48.4

48.6

48.8

49

49.2

49.4

49.6

49.8

48.5

48.8

49.5

49.8

48.5

48.8

49.5

49.8

Average

F1

1

59

Results on WebQuestions

Sim

pleGraph

CCGGraph

DepTree

UDepLambda

DepLambda

Yih15

Xu16

48.5

49

49.5

50

50.5

51

51.5

52

52.5

53

53.5

48.5 48.648.8

49.5

50.3

52.5

53.3

48.5 48.648.8

49.5

50.3

Average

F1

1

60

Learning Model

? We do not have access to gold graphs

? Access only to the answers rather than the query

? Solution: use a surrogate gold graph

Surrogate Gold Graph:

Learning Model

We do not have access to gold graphs

Access only to the answers rather than the query

Instead use a surrogate gold graph

Surrogate Gold Graph:

(g+,u+) = argmax(g,u)∈O(q)

Φ(g,u,q,KB) ·θ t

40 / 64

Oracle Graphs

61

Dependencies to Logical FormsSingle Type System


nsubj dobj

root

All constituents are of the same lambda expression typeTYPE[acquired] = TYPE[Pixar] = TYPE[(dobj acquired Pixar)]

61



nsubj dobj

root

All words have a lambda expression of type ηI TYPE[acquired] = η

I TYPE[Pixar] = η

I TYPE[(dobj acquired Pixar)] = η

=⇒ TYPE[dobj] = η → η → η

61



nsubj dobj

root

All constituents have a lambda expression of type ηI TYPE[acquired] = η

I TYPE[Pixar] = η


=⇒ TYPE[dobj] = η → η → η

61



nsubj dobj

root

All constituents have a lambda expression of type ηI TYPE[acquired] = η

I TYPE[Pixar] = η


=⇒ TYPE[dobj] = η → η → η61

Dependencies to Logical FormsLambda Calculus for Single Type System


nsubj dobj

root


acquired⇒ λxe.acquired(xe)

⇒TYPE = Event→ Bool

Pixar⇒ λxa.Pixar(xa)

⇒TYPE = Ind→ Bool

Here TYPE[acquired] 6= TYPE[Pixar] 7

61



nsubj dobj

root


acquired⇒ λxe.acquired(xe) ⇒TYPE = Event→ BoolPixar⇒ λxa.Pixar(xa) ⇒TYPE = Ind→ Bool

Here TYPE[acquired] 6= TYPE[Pixar] 7

61

Dependencies to Logical FormsLambda Calculus


nsubj dobj

root



This operation mirrors the tree structure

62



nsubj dobj

root


acquired⇒ λxaxe.acquired(xe)

⇒TYPE = Ind×Event→ Bool

Pixar⇒ λxaxe.Pixar(xa)

⇒TYPE = Ind×Event→ Bool

Here η = TYPE[acquired] = TYPE[Pixar] X

62



nsubj dobj

root


acquired⇒ λxaxe.acquired(xe) ⇒TYPE = Ind×Event→ BoolPixar⇒ λxaxe.Pixar(xa) ⇒TYPE = Ind×Event→ Bool

Here η = TYPE[acquired] = TYPE[Pixar] X

62

ConjunctionsSentence:

Eminem signed to Interscope and discovered 50 Cent.

Binarized tree:(nsubj (conj-vp (cc s to I and) d 50) Eminem)

Substitution:

conj-vp⇒ λ fgx.∃yz. f (y)∧g(z)∧ coord(x,y,z)

Logical Expression:

λw.∃xyz.Eminem(xa)∧ coord(w,y,z)∧arg1(we,xa)∧ s to I(y)∧d 50(z)

Post processing:

λe.∃xyz.Eminem(xa)∧ arg1(ye,xa)∧arg1(ze,xa)∧ s to I(y)∧d 50(z)

62

Graph Transformation: CONTRACT operation

What language do the people in Ghana speak?

language target people

x e1 y e2 Ghanaspeak.arg2

speak.arg1

people.arg1

people.nmod.in

type

type

Ungrounded graph

language.human language

target

x m Ghanalocation.country

.official language.2location.country

.official language.1

type

Grounded graph

62

Graph Mismatch: EXPAND operation

What to do Washington DC December?

Before EXPAND

I λ z.∃xyw. TARGET(xa)∧do(ze)∧ arg1(ze,xa)∧Washington DC(ya)∧December(wa)

After EXPAND

I λ z.∃xyw. TARGET(xa)∧do(ze)∧ arg1(ze,xa)∧Washington DC(ya)∧dep(ze,ya)∧December(wa)∧dep(ze,wa)

62

Dependency Graphs to Logical Forms

Anna wants to marry Kristoff

nsubj

xcomp

mark dobj

nsubj


Ω Ω

nsubj

xcomp

mark dobj

bind nsubj

63

Dependency Graphs to Logical Forms


Ω Ω

nsubj

xcomp

mark dobj

bind nsubj

Substitution ExpressionsBIND = λ f λgλx. f (x)∧g(x)xcomp = λ fgx.∃y. f (x)∧g(y)∧xcomp(xe,ye)ω = λx.EQ(x,ω)

Final Expression:

λ z.∃xyw.wants(ze)∧Anna(xa)∧ arg1(ze,xa)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ Kristoff(wa)∧ arg2(ye,wa) .

64

Linguists-defined vs. Machine-induced Natural Language ... · Tom Kwiatkowski Michael Collins Slav...

Documents

Transcript of Linguists-defined vs. Machine-induced Natural Language ... · Tom Kwiatkowski Michael Collins Slav...