Neural Networks for Discourse Coherence

Roy Aslan, Dwayne Campbell, Chris Kedzie

Task

1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

2. blah blah...4. ...5. ...1. blah blah... 3. blah… 6. ...

rank( ) rank( )

Task

1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

2. blah blah...4. ...5. ...1. blah blah... 3. blah… 6. ...

>

Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

s1 s2 s3

s2 s3 s4

s3 s4 s5

s4 s5 s6

Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

s1 s2 s3

s2 s3 s4

s3 s4 s5

s4 s5 s6

log p(y1= coherent | s1,s2,s3 )

log p(y3= coherent | s3,s4,s5 )

log p(y4= coherent | s4,s5,s3 )

log p(y2= coherent | s2,s3,s4 )

Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

s1 s2 s3

s2 s3 s4

s3 s4 s5

s4 s5 s6

log p(y1= coherent | s1,s2,s3 )

log p(y3= coherent | s3,s4,s5 )

log p(y4= coherent | s4,s5,s3 )

log p(y2= coherent | s2,s3,s4 )+

+

+

= log p( = coherent)

Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...

s1 s2 s3

s2 s3 s4

s3 s4 s5

s4 s5 s6

log p(y1= coherent | s1,s2,s3 )

log p(y3= coherent | s3,s4,s5 )

log p(y4= coherent | s4,s5,s3 )

log p(y2= coherent | s2,s3,s4 )+

+

+

= log p( = coherent) ≜ rank( )

Task

s1 s2 s3

p(Y | s1,s2,s3 ) ⇒

h1

o

Task

s1 s2 s3

p(Y | s1,s2,s3 ) ⇒

h1

o

● 3 models implemented with this framework

● models vary differ at the sentence layer

Model 1: CBOW Model

w1 w2 w3 w4 wn

Model 1: CBOW Model

w1 w2 w3 w4 wn+ + + ...( )n = s

Model 2: Recurrent Model

Model 3: Recursive Model

Results

Results

See our final paper :)

Why is this useful?Discourse might be helpful for non-factoid Question Answering.

``Empirically we show that modeling answer discourse structures is complementary to modeling lexical semantic similarity and that the best performance is obtained when they are tightly integrated.’’

Jansen, Peter, Mihai Surdeanu, and Peter Clark.

"Discourse Complements Lexical Semantics for Non-factoid Answer Reranking."

Application to QA

query

Information RetrievalSystem

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

lexical similarity

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

lexical similarity

discoursestructure

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

lexical similarity

discoursestructure

Candidate Answer Passage 1

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

lexical similarity

discoursestructure

Candidate Answer Passage 1

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 3

Candidate Answer Passage 2

Candidate Answer Passage 1

Application to QA

query

Information RetrievalSystem

Answer Re-Ranker

lexical similarity

discoursestructure

Candidate Answer Passage 1

Candidate Answer Passage 3

Candidate Answer Passage 2

Discourse Structure

Features● Explicit discourse markers

○ “because”, “however”, …

● RST Parse

Discourse Structure (NNET features)

1. blah blah... 2. blah blah...3. blah… 4. ...5. ...

s1 s2

s2 s3

s3 s4

s4 s5

query

query

query

query

Results

See our final paper :)

The End

Thanks!