Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask...
Transcript of Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask...
![Page 1: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/1.jpg)
Task QuestionTask Question
• Is it possible to monitor news media fromIs it possible to monitor news media from regions all over the world over extended periods of time, extracting low-level events from them, and piece them together to automatically track and predict conflict in all the regions of the
ld?world?
![Page 2: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/2.jpg)
The Ares projectThe Ares project
http://ares.cs.rice.edu
RiceSingularity
OnlineInformationSources
RiceEventData
Extractor
detection
Hubs &Authorities
Models
Authorities
AP, AFP,BBC, Reuters,
Over 1 millionarticles on theMiddle East from… Middle East from1979 to 2005 (filtered automatically)
![Page 3: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/3.jpg)
Analysis of wire storiesAnalysis of wire stories
Singularity detectionRelevance filter
Date Actor Target Weis Code Wies event Goldstein scale790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1790415 EGY ISR 31 (MEET) 1
Singularity detectionon aggregated eventsdata
790415 EGY ISR 31 (MEET) 1790415 ISRMIL PAL 223 (MIL ENGAGEMENT) -10790415 PALPL JOR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 193 (CUT AID) -5.6790415 IRQ EGY 31 (MEET) 1790415 EGY IRQ 31 (MEET) 1790415 ARB CHR 223 (MIL ENGAGEMENT) -10790415 JOR AUS 32 (VISIT) 1 9
Hubs and authoritiesanalysis of events
790415 JOR AUS 32 (VISIT) 1.9790415 UGA CHR 32 (VISIT) 1.9790415 ISRGOV ISRSET 54 (ASSURE) 2.8
data
![Page 4: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/4.jpg)
Embedded learner designEmbedded learner design• Representation p
– Identify relevant stories, extract event data from them, build time series models and graph-theoretic models.
L i• Learning– Identifying regime shifts in events data, tracking
evolution of militarized interstate disputes (MIDs) by p ( ) yhubs/authorities analysis of events data
• Decision-makingI i l i f tb k f MID– Issuing early warnings of outbreak of MIDs
![Page 5: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/5.jpg)
Identifying relevant storiesIdentifying relevant stories
– Only about 20% of stories contain events that are to be extracted.
• The rest are interpretations, (e.g., op-eds), or are events ( g )not about conflict (e.g., sports)
– We have trained Naïve Bayes (precision 86% and recall 81%), SVM classifiers (precision 92% and ), (precall 89%) & Okapi classifiers (precision 93% and recall 87%) using a labeled set of 180,000 stories from Reuters.
– Surprisingly difficult problem!• Lack of large labeled data sets; • Poor transfer to other sources (AP/BBC)• Poor transfer to other sources (AP/BBC)• The category of “event containing stories” is not well-
separated from others, and changes with timeLee, Tran, Singer, Subramanian, 2006
![Page 6: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/6.jpg)
Okapi classifierOkapi classifier• Reuters data set: relevant
categories are GVIO, GDIP, G13; irrelevant categories: 1POL 2ECO
RelNew article categories: 1POL, 2ECO,
3SPO, ECAT, G12, G131, GDEF, GPOL
I
New article
IrrOkapi measure takestwo articles and gives
Decision rule: sum of top N Okapi scores in Rel set > f k
two articles and givesthe similarity between them.
sum of top N Okapi scores in Irr setthen classify as rel; else irr
![Page 7: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/7.jpg)
Event extractionEvent extraction
![Page 8: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/8.jpg)
Parse sentenceParse sentence
Klein and Manning parser
![Page 9: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/9.jpg)
Pronoun de-referencingPronoun de referencing
![Page 10: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/10.jpg)
S t f t tiSentence fragmentation
Correlative conjunctions
Extract embedded sentences (SBAR)
![Page 11: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/11.jpg)
Conditional random fieldsConditional random fields
We extract who (actor) did what (event) to whom (target)
Not exactly the same as NER
![Page 12: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/12.jpg)
ResultsResults
TABARIis stateof the artcodercoderin politicalscience
200 Reuters sentences; hand-labeled with actor, target,and event codes (22 and 02).
Stepinksi, Stoll, Subramanian 2006
![Page 13: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/13.jpg)
Events dataEvents data
Date Actor Target Weis Code Wies event Goldstein scale790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) 10790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1790415 EGY ISR 31 (MEET) 1790415 ISRMIL PAL 223 (MIL ENGAGEMENT) -10790415 PALPL JOR 223 (MIL ENGAGEMENT) 10790415 PALPL JOR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 193 (CUT AID) -5.6790415 IRQ EGY 31 (MEET) 1790415 EGY IRQ 31 (MEET) 1790415 ARB CHR 223 (MIL ENGAGEMENT) -10790415 JOR AUS 32 (VISIT) 1 9
177 336 t f A il 1979 t O t b 2003 i L t
790415 JOR AUS 32 (VISIT) 1.9790415 UGA CHR 32 (VISIT) 1.9790415 ISRGOV ISRSET 54 (ASSURE) 2.8
177,336 events from April 1979 to October 2003 in Levantdata set (KEDS).
![Page 14: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/14.jpg)
What can be predicted?What can be predicted?
![Page 15: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/15.jpg)
Singularity detection
Stoll and Subramanian, 2004, 2006
![Page 16: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/16.jpg)
Singularities = MID start/endSingularities MID start/end
biweek Date range
event
17-35 11/79 to 8/80
Start of Iran/Iraq war
105-111 4/83 to 7/83
Beirut suicide attack, end of Iran/Iraq war
244 1/91 t 2/91 D t St244 1/91 to 2/91 Desert Storm
413-425 1/95 to 7/95 Rabin assassination/start of Intifada
483-518 10/97 to 2/99
US/Iraq confrontation via Richard Butler/arms inspectors
522-539 4/99 to Second intifada Israel/Palestine522 539 4/99 to 11/99
Second intifada Israel/Palestine
![Page 17: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/17.jpg)
Interaction graphsInteraction graphs
• Model interactions between countries in aModel interactions between countries in a directed graph.
Date Actor Target Weis Code Wies event Goldstein scale790415 ARB ISR 223 (MIL ENGAGEMENT) -10790415 EGY AFD 194 (HALT NEGOTIATION) -3.8790415 PALPL ISR 223 (MIL ENGAGEMENT) -10790415 UNK ISR 223 (MIL ENGAGEMENT) -10790415 ISR EGY 31 (MEET) 1
ARB ISRARB ISR
EGY UNK
AFD PALPL
![Page 18: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/18.jpg)
Hubs and authorities for events data
• A hub node is an important initiator of events.p• An authority node is an important target of
events.• Hypothesis:
– Identifying hubs and authorities over a particular temporal chunk of events data tells us who the keytemporal chunk of events data tells us who the key actors and targets are.
– Changes in the number and size of connected t i th i t ti h i l t ti lcomponents in the interaction graph signal potential
outbreak of conflict.
![Page 19: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/19.jpg)
Hubs/Authorities picture of Iran Iraq war
![Page 20: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/20.jpg)
2 weeks prior to Desert Storm2 weeks prior to Desert Storm
![Page 21: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/21.jpg)
Validation using MID dataValidation using MID data• Number of bi-weeks with MIDS in Levant data: 41 out of
589.• Result 1: Hubs and Authorities correctly identify actors
and targets in impending conflict.and targets in impending conflict.• Result 2: Simple regression model on change in hubs
and authorities scores, change in number of connected components change in size of largest component 4components, change in size of largest component 4 weeks before MID, predicts MID onset.
• Problem: false alarm rate of 16% can be reduced by ddi liti l k l d f fli tadding political knowledge of conflict.
Stoll and Subramanian, 2006
![Page 22: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/22.jpg)
![Page 23: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/23.jpg)
Current workCurrent work
• Extracting economic events along withExtracting economic events along with political events to improve accuracy of prediction of both economic and politicalprediction of both economic and political events.
![Page 24: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/24.jpg)
PublicationsPublications
A OKAPI b d h f ti l filt i L Th St ll S b i 2006 Ri• An OKAPI-based approach for article filtering, Lee, Than, Stoll, Subramanian, 2006 Rice University Technical Report.
• Hubs, authorities and networks: predicting conflict using events data, R. Stoll and D. Subramanian, International Studies Association, 2006 (invited paper).
• Events patterns and analysis D Subramanian and R Stoll in Programming for Peace:• Events, patterns and analysis, D. Subramanian and R. Stoll, in Programming for Peace: Computer-aided methods for international conflict resolution and prevention, 2006, Springer Verlag, R. Trappl (ed).
• Four Way Street? Saudi Arabia's Behavior among the superpowers, 1966-1999, R. Stoll and D. Subramanian, James A Baker III Institute for Public Policy Series, 2004. , y ,
• Events, patterns and analysis: forecasting conflict in the 21st century, R. Stoll and D. Subramanian, Proceedings of the National Conference on Digital Government Research, 2004.
• Forecasting international conflict in the 21st century, D. Subramanian and R. Stoll, in Proc. of the Symposium on Computer-aided methods for international conflict resolution, 2002.
![Page 25: Task QuestionTask Question - Rice Universitydevika/conflict/papers/stolltalk.pdfTask QuestionTask Question • Is it possible to monitor news media fromIs it possible to monitor news](https://reader035.fdocuments.us/reader035/viewer/2022081406/5f1096777e708231d449d837/html5/thumbnails/25.jpg)
The research teamThe research team