Concentric Semantic Snapshot
-
Upload
jose-luis-redondo-garcia -
Category
Engineering
-
view
616 -
download
0
Transcript of Concentric Semantic Snapshot
THE CONCENTRIC NATURE OF NEWS SEMANTIC SNAPSHOTS
JOSÉ LUIS REDONDO GARCIAGIUSEPPE RIZZORAPHAËL TRONCY
@peputo / [email protected]
@giusepperizzo / [email protected]
@rtroncy / [email protected]
2
Overview
May 1, 2023 8th International Conference on Knowledge Capture
1. Introducing the Problem: Contextualizing News Items o The News Semantic Snapshot (NSS)
2. Previous Work:o Frequency-based Functionso Multidimensional Relevancy Approach
3. A Concentric Model for Generating NSS
3
Overview
May 1, 2023 8th International Conference on Knowledge Capture
1. Introducing the Problem: Contextualizing News Items o The News Semantic Snapshot (NSS)
2. Previous Work:o Frequency-based Functionso Multidimensional Relevancy Approach
3. A Concentric Model for Generating NSS
4
The Problem: Contextualizing News
May 1, 2023 8th International Conference on Knowledge Capture
Wolfgang Schäuble
Finance Minister Ruling Party in Ger.
Christian Democratic Union
1 2 3
5May 1, 2023 8th International Conference on Knowledge Capture
Sarah Harrison
WikiLeaks Editor Airport in Moscow
Sheremetyevo
The Problem: Contextualizing News1 2 3
6
Contextualizing News: Applications
May 1, 2023 8th International Conference on Knowledge Capture
1 2 3
7
1 2 3
News Semantic Snapshot (NSS) [1]
May 1, 2023 8th International Conference on Knowledge Capture
News Semantic Snapshot (NSS)
[1] Redondo et al., Generating the Semantic Snapshot of Newscasts using Entity Expansion, ICWE 2015, Rotterdam.
May 1, 2023 8
Recreating the NSS
News Semantic Snapshot8th International Conference on Knowledge Capture
ea eb ec ed ef eg eh ei ej ek el em
ea ec eh ej ek em
(2) SELECTION: filtering, clustering, ranking…
(1) EXPANSION: query generation, search, document retrieval…
ea eb ec ed
1 2 3
May 1, 2023 9
Involving: (experts in the news domain + users)Dimensions:
Play with the data and help us to extend it at: https://github.com/jluisred/NewsConceptExpansion/wiki/Golden-Standard-Creation
News Semantic Snapshot: Gold Standard
(1) Video Subtitles(2) Image in the video(3) Text in the video image(4) Suggestions of an expert(5) Related articles
8th International Conference on Knowledge Capture
1 2 3
May 1, 2023 10
Recreating the NSS
News Semantic Snapshot8th International Conference on Knowledge Capture
ea eb ec ed ef eg eh ei ej ek el em
ea ec eh ej ek em
(2) SELECTION: filtering, clustering, ranking…
(1) EXPANSION: query generation, search, document retrieval…
ea eb ec ed
1 2 3
May 1, 2023
(1) Bringing in Missing Entities: News Entity Expansion
11
1.a)
8th International Conference on Knowledge Capture
Web sites to be crawled:- Google- L1 : A set of 10
internationals English speaking newspapers
- L2 : A set of 3 international newspapers used in GS
Temporal Window:- 1W: - 2W: Annotation filtering- Schema.org
1.b)Parameters [1]:
1 2 3
[1] Redondo et al., Generating the Semantic Snapshot of Newscasts using Entity Expansion, ICWE 2015, Rotterdam.
May 1, 2023 12
News Semantic Snapshot8th International Conference on Knowledge Capture
ea eb ec ed ef eg eh ei ej ek el em
ea ec eh ej ek em
(2) SELECTION: filtering, clustering, ranking…
(1) EXPANSION: query generation, search, document retrieval…
ea eb ec ed
Recall (E. Expansion) = 0.91
Recall (NER on Subtitles) = 0.42
Recreating the NSS1 2 3
May 1, 2023 138th International Conference on Knowledge Capture
(NSS)
(Entity Expansion)
0
N
FIdeal(ei)
(NSS)
FX(ei)
=?MNDCG
The Selection Problem: 1 2 3
14
Overview
May 1, 2023 8th International Conference on Knowledge Capture
1. Introducing the Problem: Contextualizing News Items o The News Semantic Snapshot (NSS)
2. Previous Work:o Frequency-based Functiono Multidimensional Relevancy Approach
3. A Concentric Model for Generating NSS
May 1, 2023 15
1º Entity Frequency SNOW Workshop 2014 [2]
8th International Conference on Knowledge Capture
A
1 2 3
[2] Redondo et al., Describing and Contextualizing Events in TV News Show}, SNOW Workshop, WWW 2014, Seoul, Korea.
May 1, 2023 16
Frequency Based: Results
8th International Conference on Knowledge Capture
(NSS)
(Expansion)
FREQ0
N
(NSS
)
F(Laura Poitras) = 2
F(Glenn Greenwald) = 1
1 2 3
May 1, 2023 15th International Conference on Web Engineering (ICWE) 17
(Fr) (FrGaussian)
Multidimensional ApproachICWE 2015 [1]
1 2 3
[1] Redondo et al., Generating the Semantic Snapshot of Newscasts using Entity Expansion, ICWE 2015, Rotterdam.
May 1, 2023
POPULARITY (FPOP) EXPERT RULES (FEXP)
18
- Based on Google Trends- w = 2 months- μ + 2*σ (2.5%)
Example:- [ Location, = 0.48 ]- [ Person, = 0.74 ]- [ Organization, = 0.95 ]- [ < 2 , = 0.0 ]
15th International Conference on Web Engineering (ICWE) 18
Multidimensional Approach1 2 3
May 1, 2023 19
- News Entity Expansion + Dimensions Generate the News Semantic Snapshot
- Best score: 0.667 in MNDCG at 10, better than BS1/2
• Collection: CSE (Google + 2W + Schema.org)• Ranking:
• Expert Rules• Popularity
8th International Conference on Knowledge Capture
Multidimensionality: Results1 2 3
May 1, 2023 208th International Conference on Knowledge Capture
(NSS))
(Expansion)
FREQ POP EXP
+ + =
(NSS
)
Multidimensionality: Results1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 21
Follow up: Fine-Tuning
1. Exploit Google Relevance (+1.80%)2. Promote Subtitle Entities (+2.50%)3. Exploit Named Entity Extractor’s confidence (+0.20%)4. Interpret popularity Dimension (+1.40%)5. Performing Clustering before Filtering (-0.60%)
- NO SIGNIFICANT IMPROVEMENT -
1 2 3
May 1, 2023 228th International Conference on Knowledge Capture
(NSS)
TuneFunction XFREQ POP EXP
No Improvement: Why?Re-ShuffleOriginal
(NSS
)
How many Dimensions?How to combine them?
1 2 3
23
Overview
May 1, 2023 8th International Conference on Knowledge Capture
1. Introducing the Problem: Contextualizing News Items o The News Semantic Snapshot (NSS)
2. Previous Work:o Frequency-based Functiono Multidimensional Relevancy Approach
3. A Concentric Model for Generating NSS
May 1, 2023 8th International Conference on Knowledge Capture 24
Thinking Outside the Box:
1. Is there room for improvement?2. Is MNDCG a good measure to
evaluate NSS? 3. How to significantly improve the
approach?
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 25
Room for Improvement?
GAIN
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 26
Room for Improvement?1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 27
How to Evaluate NSS? MNDCG:• Too focused on success at first positions (decay
Function)
• NSS intends to be flexible, ranking is application-dependent
COMPACTNESS:• Prioritizes coverage over ranking• Compromise between: Recall and NSS size• Recall*: positives are weighted according to score in GT
(NSS)
1 2 3
May 1, 2023 288th International Conference on Knowledge Capture
Compactness:Recall: 22/33 = 0.66
Sa = 27
Sb = 33
Sc = 54
Sa = 27
Sb = 33
Sc= 54
(NSS
)
A B CA
B
C
> >
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 29
Re-thinking the Approach: Concentric Snapshot
Duality in News Entity Spectrum:• REPRESENTATIVE entities:
• Driving the plot of the story, sometimes evident for users. • RELEVANT entities
• Related to former via specific reasons
Exploit the entity semantic relations
Popular?
Suggested by Expert?
Informative?Unexpected?
Interesting?
Explicative?Highly
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 30
Hypothesis: Concentric SnapshotCORE:• Representative entities
• Spottable via Frequency dimensions
• High degree of cohesiveness
CRUST:• Attached to the Core via
particular relations
• Agnostic to relevancy nature: informativeness, interestingness, etc.
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 31
Core Generationa) Representative entities: Frequency Dimension
(NSS)
b) Cohesiveness (DBpedia)
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 32
Crust Generation
The number of Web documents talking simultaneously about a particular entity e and the Core:
??
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 33
Experimental Settings
1. Entity Frequency• Core1: Jaro-Winkler > 0.9 • Core2: Frequency based on Exact String matching
2. Cohesiveness: • Everything is Connected Engine [3]• Skb(e1, e2) > 0.125
CORE: (2 configurations)
[3] Everything is Connected Engine:
https://github.com/mmlab/eice
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 34
1. Candidates for CRUST generation: • Ex1: 1° ICWE2015 by R*(50): L2+Google, F3 1W, Gauss+ POP• Ex2: 2° ICWE 2015 by R*(50): L2+Google, F3 1W, Freq + POP
2. Function for attaching entities to CORE:• SWEB(ei, Core) over Google CSE, default Configuration
CRUST:
Experimental Settings1 2 3
(2 configurations)
May 1, 2023 8th International Conference on Knowledge Capture 35
• Core+Crust: • CrustOnly:
Projecting CORE and CRUST:
(NSS)
(Expansion)
CORE CRUST Core+Crust CrustOnly
Experimental Settings1 2 3
(2 configurations)
May 1, 2023 8th International Conference on Knowledge Capture 36
Baselines:
BAS01: best run in ICWE 2015 at R*(50)BAS02: second best run in ICWE 2015 at R*(50)
FREQPOPEXP
Experimental Settings1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 37
Results: Compactness
Percentage decrease of 36.9% over BAS01
IdealGT: size of SSN according to Gold Standard
(2*2*2 + 2) Runs
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 38
Results: Recall* over N1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 39
Conclusion• News applications can benefit from the News Semantic Snapshot (NSS)
• Proposed a concentric based model for generating the NSS:• Formalizes duality in entities (Representative VS Relevant)• Exploit the entity semantic relations between Core and Crust.• Accommodate into a single model different relevancy dimensions via the notion of
web presence ( SWeb )
• Concentric model better reproduces the NSS:• Better Compactness: 36.9% over BAS01• Similar recall, Smaller size
• Concentric model easier to implement:
• Core can be reproduced via Frequency Dimension• Crust brings up relevant entities without having to deal with fuzzy dimensions
1 2 3
May 1, 2023 8th International Conference on Knowledge Capture 40
Future• Extend the number of videos considered in GT:
From 5 to 23 (+18), check [4] for more information
• Spot not only relationships between Crust and the Core but also predicates that characterize them:
[4] https://github.com/jluisred/NewsConceptExpansion/wiki/Golden-Standard-Creation
Editor in WikiLeaks
1 2 3
JOSÉ LUIS REDONDO GARCIAGIUSEPPE RIZZORAPHAËL TRONCY
@peputo / [email protected]
@giusepperizzo / [email protected]
@rtroncy / [email protected]
http://www.slideshare.net/joseluisredondo/concentric-semantic-snapshot
Visit poster at booth:
34