Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event...
Transcript of Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event...
![Page 1: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/1.jpg)
Model Combination for Event Extractionin BioNLP 2011
Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b
Andrew McCallum,a and Christopher D. Manningb
aUniversity of Massachusetts at Amherst and bStanford University
BioNLP 2011 — June 24th, 2011
1
![Page 2: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/2.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 3: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/3.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 4: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/4.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 5: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/5.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 6: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/6.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.
I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 7: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/7.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do we
I Stacked model’s output as features in stacking model
2
![Page 8: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/8.jpg)
Previous work / Motivation
I BioNLP 2009: model combination led to 4% F1improvement over best individual system (Kim et al., 2009)
I Netflix challenge: winning entry relies on modelcombination (Bennett et al., 2007)
I CoNLL 2007: winning entry relies on modelcombination (Hall et al., 2007)
I CoNLL 2003: winning entry relies on modelcombination (Florian et al., 2003)
I etc. etc. etc.I Most of these use stacking—so do weI Stacked model’s output as features in stacking model
2
![Page 9: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/9.jpg)
Stacking Model
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
10
10
�
0.2−1.2
−2.31.5
s( )= -0.1s( )= 3.2s( )= 0.5Phosphor.
Regulation
Binding
None
Theme
Cause
s( )=0.2s( )=1.3
s( )=-2.2
Maximize
s (e,a) =�
i
s (ei) +�
i,j
s (ai,j)s (e,a) =�
i
s (ei) +�
i,j
s (ai,j)
phosphorylation of TRAF2 inhibits binding to the CD40 domain
s (e,a,b) =�
i
s (ei) +�
i,j
s (ai,j) +�
p,q
s (bp,q)s( )=-2.2s( )=3.2
underglobal constrains
3
![Page 10: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/10.jpg)
Scores
s( )= 3.2Regulation
e = Reg
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
e = Reg and w = ”inhibit”
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
4
![Page 11: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/11.jpg)
Stacked Featuress (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
s( )= 3.2Regulation
e = Reg
e = Reg and w = ”inhibit”
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
e = Reg and y = Reg
5
![Page 12: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/12.jpg)
Stacked model
I Stanford Event Parsing system
I Recall: Four different decoders:(1st, 2nd-order features) × (projective, non-projective)
I Only used the parser for stacking (1-best outputs)
I Different segmentation/tokenization
I Different trigger detection
6
![Page 13: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/13.jpg)
Performance of individual components
System F1UMass 54.8
Stanford (1N) 49.9Stanford (1P) 49.0Stanford (2N) 46.5Stanford (2P) 49.5
(Genia development section, Task 1)
7
![Page 14: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/14.jpg)
Performance of individual components
System F1UMass 54.8Stanford (1N) 49.9Stanford (1P) 49.0Stanford (2N) 46.5Stanford (2P) 49.5
(Genia development section, Task 1)
7
![Page 15: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/15.jpg)
Performance of individual components
System F1 with rerankerUMass 54.8 —Stanford (1N) 49.9 50.2Stanford (1P) 49.0 49.4Stanford (2N) 46.5 47.9Stanford (2P) 49.5 50.5
(Genia development section, Task 1)
8
![Page 16: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/16.jpg)
Model combination strategies
System F1UMass 54.8Stanford (2P, reranked) 50.5
Stanford (all, reranked) 50.7UMass←2N 54.9UMass←1N 55.6UMass←1P 55.7UMass←2P 55.7UMass←all
(FAUST)
55.9
(Genia development section, Task 1)
9
![Page 17: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/17.jpg)
Model combination strategies
System F1UMass 54.8Stanford (2P, reranked) 50.5Stanford (all, reranked) 50.7
UMass←2N 54.9UMass←1N 55.6UMass←1P 55.7UMass←2P 55.7UMass←all
(FAUST)
55.9
(Genia development section, Task 1)
9
![Page 18: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/18.jpg)
Model combination strategies
System F1UMass 54.8Stanford (2P, reranked) 50.5Stanford (all, reranked) 50.7UMass←2N 54.9UMass←1N 55.6UMass←1P 55.7UMass←2P 55.7
UMass←all
(FAUST)
55.9
(Genia development section, Task 1)
9
![Page 19: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/19.jpg)
Model combination strategies
System F1UMass 54.8Stanford (2P, reranked) 50.5Stanford (all, reranked) 50.7UMass←2N 54.9UMass←1N 55.6UMass←1P 55.7UMass←2P 55.7UMass←all
(FAUST)
55.9
(Genia development section, Task 1)
9
![Page 20: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/20.jpg)
Model combination strategies
System F1UMass 54.8Stanford (2P, reranked) 50.5Stanford (all, reranked) 50.7UMass←2N 54.9UMass←1N 55.6UMass←1P 55.7UMass←2P 55.7UMass←all (FAUST) 55.9
(Genia development section, Task 1)
9
![Page 21: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/21.jpg)
Ablation analysis for stacking
System F1UMass 54.8Stanford (2P, reranked) 50.5UMass←all 55.9
UMass←all (triggers) 54.9UMass←all (arguments) 55.1
(Genia development section, Task 1)
10
![Page 22: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/22.jpg)
Ablation analysis for stacking
System F1UMass 54.8Stanford (2P, reranked) 50.5UMass←all 55.9UMass←all (triggers) 54.9UMass←all (arguments) 55.1
(Genia development section, Task 1)
10
![Page 23: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/23.jpg)
Conclusions
I Stacking: easy, effective method of model combination
I ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoder
I Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 24: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/24.jpg)
Conclusions
I Stacking: easy, effective method of model combinationI ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoder
I Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 25: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/25.jpg)
Conclusions
I Stacking: easy, effective method of model combinationI ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoder
I Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 26: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/26.jpg)
Conclusions
I Stacking: easy, effective method of model combinationI ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoder
I Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 27: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/27.jpg)
Conclusions
I Stacking: easy, effective method of model combinationI ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoderI Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 28: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/28.jpg)
Conclusions
I Stacking: easy, effective method of model combinationI ...even if base models differ significantly in performance
I Variability in models critical for success
I Tree structure best provided by projective decoderI Incorporated in UMass model via 2P stacking
I Future work: Incorporate projectivity constraint directly
Questions?
11
![Page 29: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/29.jpg)
Backup slides
12
![Page 30: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/30.jpg)
Stacked Featuress (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
s( )= 3.2Regulation
e = Reg
e = Reg and w = ”inhibit”
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
e = Reg and y = Reg
13
![Page 31: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/31.jpg)
Conjoined Features
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
s( )= 3.2Regulation
e = Reg
e = Reg and w = ”inhibit”
s (e,a,b) =�
i
si (ei) +�
i,j
si,j (ai,j) +�
p,q
sp,q (bp,q)
si (ei) =
1
1
� −2.1
1.3
e = Reg
e = Reg ∧ w =
si (ei) =
11
1
�
−2.11.2
1.3
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =
si (ei) =
11
11
�
−2.11.2
1.33.2
e = Reg
e = Reg ∧ y = Reg
e = Reg ∧ w =e = Reg ∧ w = ∧ y = Reg
si (ei) =
1
1
�
0.2
1.52.3
e = Reg
e = Reg ∧ w = ∧ = Reg
e = Reg ∧ = Reg
e = Reg and y = Reg
e = Reg and w = ”inhibit” and y = Reg
14
![Page 32: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/32.jpg)
Results on Genia
System Simple Binding Regulation TotalUMass 74.7 47.7 42.8 54.8Stanford 1N 71.4 38.6 32.8 47.8Stanford 1P 70.8 35.9 31.1 46.5Stanford 2N 69.1 35.0 27.8 44.3Stanford 2P 72.0 36.2 32.2 47.4UMass←All 76.9 43.5 44.0 55.9UMass←1N 76.4 45.1 43.8 55.6UMass←1P 75.8 43.1 44.6 55.7UMass←2N 74.9 42.8 43.8 54.9UMass←2P 75.7 46.0 44.1 55.7UMass←All (triggers) 76.4 41.2 43.1 54.9UMass←All (arguments) 76.1 41.7 43.6 55.1
15
![Page 33: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/33.jpg)
Results on Infectious Diseases
System Rec Prec F1UMass 46.2 51.1 48.5Stanford 1N 43.1 49.1 45.9Stanford 1P 40.8 46.7 43.5Stanford 2N 41.6 53.9 46.9Stanford 2P 42.8 48.1 45.3UMass←All 47.6 54.3 50.7UMass←1N 45.8 51.6 48.5UMass←1P 47.6 52.8 50.0UMass←2N 45.4 52.4 48.6UMass←2P 49.1 52.6 50.7UMass←2P (conjoined) 48.0 53.2 50.4
16
![Page 34: Model Combination for Event Extraction in BioNLP 2011€¦ · Model Combination for Event Extraction in BioNLP 2011 Sebastian Riedel,a David McClosky,b Mihai Surdeanu,b Andrew McCallum,a](https://reader034.fdocuments.us/reader034/viewer/2022051815/603d3be1678e190e6c56749e/html5/thumbnails/34.jpg)
Results on test
UMass UMass←AllRec Prec F1 Rec Prec F1
GE (Task 1) 48.5 64.1 55.2 49.4 64.8 56.0GE (Task 2) 43.9 60.9 51.0 46.7 63.8 53.9EPI (Full task) 28.1 41.6 33.5 28.9 44.5 35.0EPI (Core task) 57.0 73.3 64.2 59.9 80.3 68.6ID (Full task) 46.9 62.0 53.4 48.0 66.0 55.6ID (Core task) 49.5 62.1 55.1 50.6 66.1 57.3
17