1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire...
-
Upload
brianna-tate -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire...
![Page 1: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/1.jpg)
1
Determining the Hierarchical Structure of Perspective and Speech Expressions
Eric Breck and Claire CardieCornell University
Department of Computer Science
![Page 2: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/2.jpg)
Cornell University Computer Science COLING 2004 2
Events in the News
![Page 3: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/3.jpg)
Cornell University Computer Science COLING 2004 3
Reporting events
![Page 4: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/4.jpg)
Cornell University Computer Science COLING 2004 4
Reporting in text
Clapp sums up the environmental movement’s reaction: “The polluters are unreasonable’’
Charlie was angry at Alice’s claim that Bob was unhappy
![Page 5: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/5.jpg)
Cornell University Computer Science COLING 2004 5
Perspective and Speech Expressions (pse’s)
A perspective expression is text denoting an explicit opinion, belief, sentiment, etc. The actor was elated that … John’s firm belief in …
A speech expression is text denoting spoken or written communication … argued the attorney ... … the 9/11 Commission’s final report …
![Page 6: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/6.jpg)
Cornell University Computer Science COLING 2004 6
Grand Vision
angry
claim
(implicit)
unhappy
Charlie was angry at Alice’s claim that Bob was unhappy
writer
Charlie Alice
Bob
that Bob was unhappy
![Page 7: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/7.jpg)
Cornell University Computer Science COLING 2004 7
This Work
angry
claim
(implicit)
unhappy
![Page 8: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/8.jpg)
Cornell University Computer Science COLING 2004 8
System Output: Pse Hierarchy
Charlie was angry at Alice’s claim that Bob was unhappy
78% accurate!
angry
claim
(implicit)
unhappy
![Page 9: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/9.jpg)
Cornell University Computer Science COLING 2004 9
Related Work: Abstract
Bergler, 1993 Lexical semantics of reporting verbs
Gerard, 2000 Abstract model of news reader
![Page 10: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/10.jpg)
Cornell University Computer Science COLING 2004 10
Related Work: Concrete
Bethard et al., 2004 Extract propositional opinions & holders
Wiebe, 1994 Tracks “point of view” in narrative text
Wiebe et al., 2003 Preliminary results on pse identification
Gildea and Jurafsky, 2002 Semantic Role ID - use for finding sources?
![Page 11: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/11.jpg)
Cornell University Computer Science COLING 2004 11
Baseline 1: Only filter through writer
Only 66% correct
angry
claim
(implicit)
unhappy
unhappy
unhappy
![Page 12: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/12.jpg)
Cornell University Computer Science COLING 2004 12
Baseline 2: Dependency Tree
72% correct
angry
(implicit)
claim
unhappy
claim
unhappy
claim
unhappy
![Page 13: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/13.jpg)
Cornell University Computer Science COLING 2004 13
A Learning Approach How do we cast the recovery of
hierarchical structure as a learning problem?
Simplest solution Learn pairwise attachment decisions
Is pseparent the parent of psetarget? Combine decisions to form tree
Other solutions are possible (n-ary decisions, tree-modeling, etc.)
![Page 14: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/14.jpg)
Cornell University Computer Science COLING 2004 14
Training instances
angry
claim
(implicit)
unhappy
![Page 15: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/15.jpg)
Cornell University Computer Science COLING 2004 15
Training instances
<unhappy, (implicit)>
angry
claim
(implicit)
unhappy
![Page 16: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/16.jpg)
Cornell University Computer Science COLING 2004 16
Training instances
<unhappy, (implicit)><claim, (implicit)>
angry
claim
(implicit)
unhappy
![Page 17: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/17.jpg)
Cornell University Computer Science COLING 2004 17
Training instances
<unhappy, (implicit)><claim, (implicit)><angry, (implicit)>
angry
claim
(implicit)
unhappy
![Page 18: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/18.jpg)
Cornell University Computer Science COLING 2004 18
Training instances
<unhappy, (implicit)><claim, (implicit)><angry, (implicit)><unhappy, claim><claim, unhappy>
angry
claim
(implicit)
unhappy
![Page 19: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/19.jpg)
Cornell University Computer Science COLING 2004 19
Training instances
<unhappy, (implicit)><claim, (implicit)><angry, (implicit)><unhappy, claim><claim, unhappy>
angry
claim
(implicit)
unhappy <unhappy,
angry><angry,
unhappy>
![Page 20: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/20.jpg)
Cornell University Computer Science COLING 2004 20
Training instances
<unhappy, (implicit)><claim, (implicit)><angry, (implicit)><unhappy, claim><claim, unhappy>
angry
claim
(implicit)
unhappy<unhappy, angry>
<angry, unhappy> <angry, claim><claim, angry>
![Page 21: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/21.jpg)
Cornell University Computer Science COLING 2004 21
Decision Combination(implicit)
angryclaim
unhappy
![Page 22: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/22.jpg)
Cornell University Computer Science COLING 2004 22
Decision Combination
angry
(implicit) angry
0.9 <angry, (implicit)>
0.1 <angry, claim>
0.1 <angry, unhappy>
claim
unhappy
![Page 23: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/23.jpg)
Cornell University Computer Science COLING 2004 23
Decision Combination
angry
(implicit)
claim
unhappy
![Page 24: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/24.jpg)
Cornell University Computer Science COLING 2004 24
Decision Combination
angryclai
m
(implicit) claim
0.5 <claim, (implicit)>
0.4 <claim, angry>
0.3 <claim, unhappy>unhapp
y
![Page 25: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/25.jpg)
Cornell University Computer Science COLING 2004 25
Decision Combination
angry
claim
(implicit)
unhappy
![Page 26: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/26.jpg)
Cornell University Computer Science COLING 2004 26
Decision Combination
angry
claim
(implicit)
unhappy
unhappy
0.7 <unhappy, claim>
0.5 <unhappy, (implicit)>
0.2 <unhappy, angry>
![Page 27: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/27.jpg)
Cornell University Computer Science COLING 2004 27
Decision Combination
angry
claim
(implicit)
unhappy
![Page 28: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/28.jpg)
Cornell University Computer Science COLING 2004 28
Features(1)
All features based on error analysis
Parse-based features Domination+ variants
Positional features Relative position of pseparent and psetarget
![Page 29: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/29.jpg)
Cornell University Computer Science COLING 2004 29
Features(2) Lexical features
writer’s implicit pse “said” “according to” part of speech
Genre-specific features Charlie, she noted, dislikes Chinese
food. “Alice disagrees with me,” Bob said.
![Page 30: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/30.jpg)
Cornell University Computer Science COLING 2004 30
Resources
GATE toolkit (Cunningham et al, 2002) - part-of-speech, tokenization, sentence boundaries
Collins parser (1999) - extracted dependency parses
CASS partial parser (Abney, 1997) IND decision trees (Buntine, 1993)
![Page 31: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/31.jpg)
Cornell University Computer Science COLING 2004 31
Data From the NRRC Multi-Perspective
Question Answering workshop (Wiebe, 2002)
535 newswire documents (66 for development, 469 for evaluation)
All pse’s annotated, along with sources and other information Hierarchical pse structure annotated
for each sentence*
![Page 32: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/32.jpg)
Cornell University Computer Science COLING 2004 32
Example (truncated) model
One learned tree, truncated to depth 3: pse0 is parent of pse1 iff
pse0 is (implicit) And pse1 is not in quotes
OR pse0 is said
Typical trees on development data: Depth ~20, ~700 leaves
![Page 33: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/33.jpg)
Cornell University Computer Science COLING 2004 33
Evaluation
Dependency-based metric (Lin, 1995) Percentage of pse’s whose parents are
identified correctly
Percentage of sentences with perfectly identified structure
Performance of binary classifier
![Page 34: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/34.jpg)
Cornell University Computer Science COLING 2004 34
Results
6672
78
3645
55
73 7882
0
10
20
30
40
50
60
70
80
90
DependencyScore
SentencesPerfect
BinaryClassifier
Baseline 1 -writer parent
Baseline 2 -syntacticdomination
Decision Tree
![Page 35: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/35.jpg)
Cornell University Computer Science COLING 2004 35
Error Analysis
Pairwise decisions prevent the model from learning larger structure
Speech events and perspective expressions behave differently
Treebank-style parses don’t always have the structure we need
![Page 36: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/36.jpg)
Cornell University Computer Science COLING 2004 36
Future Work
Identify pse’s Identify sources Evaluate alternative structure-
learning methods Use the structure to generate
perspective-oriented summaries
![Page 37: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/37.jpg)
Cornell University Computer Science COLING 2004 37
Conclusions
Understanding pse structure is important for understanding text
Automated analysis of pse structure is possible
![Page 38: 1 Determining the Hierarchical Structure of Perspective and Speech Expressions Eric Breck and Claire Cardie Cornell University Department of Computer Science.](https://reader036.fdocuments.us/reader036/viewer/2022081519/56649ec55503460f94bd083b/html5/thumbnails/38.jpg)
Cornell University Computer Science COLING 2004 38
Thank you!