Impact of Linguistic Analysis on the Semantic Impact of Linguistic Analysis on the Semantic Graph Coverage and Learning of Document Graph Coverage and Learning of Document ExtractsExtracts

Jurij LeskovecJurij LeskovecCarnegie Mellon Univ.Carnegie Mellon Univ.Pittsburgh, PAPittsburgh, PA

Document SummarizationDocument Summarization

AAAI ConferenceAAAI Conference, , PittsburghPittsburghJuly 11, 2005July 11, 2005

Marko GrobelnikMarko GrobelnikJozef Stefan InstituteJozef Stefan InstituteLjubljana, Slovenia Ljubljana, Slovenia

Natasa Milic-Natasa Milic-FraylingFraylingMicrosoft ResearchMicrosoft ResearchCambridge, UKCambridge, UK

Problem definitionProblem definition

Produce a shorter version of the Produce a shorter version of the original document by original document by selecting selecting sentencessentences from the text from the text

HypothesisHypothesis::

Extracted summaries should capture Extracted summaries should capture prominent concepts and relations in the text.prominent concepts and relations in the text.

Structure of the semantic graph of a Structure of the semantic graph of a document could help identify the key document could help identify the key concepts and relations for summarization.concepts and relations for summarization.

Cracks Appear in U.N. Trade Embargo Against Iraq.

Cracks appeared Tuesday in the U.N. trade embargo against Iraq as Saddam Hussein sought to circumvent the economic noose around his country. Japan, meanwhile, announced it would increase its aid to countries hardest hit by enforcing the sanctions. Hoping to defuse criticism that it is not doing its share to oppose Baghdad, Japan said up to $2 billion in aid may be sent to nations most affected by the U.N. embargo on Iraq. President Bush on Tuesday night promised a joint session of Congress and a nationwide radio and television audience that ``Saddam Hussein will fail'' to make his conquest of Kuwait permanent. ``America must stand up to aggression, and we will,'' said Bush, who added that the U.S. military may remain in the Saudi Arabian desert indefinitely. ``I cannot predict just how long it will take to convince Iraq to withdraw from Kuwait,'' Bush said. More than 150,000 U.S. troops have been sent to the Persian Gulf region to deter a possible Iraqi invasion of Saudi Arabia. Bush's aides said the president would follow his address to Congress with a televised message for the Iraqi people, declaring the world is united against their government's invasion of Kuwait. Saddam had offered Bush time on Iraqi TV. The Philippines and Namibia, the first of the developing nations to respond to an offer Monday by Saddam of free oil _ in exchange for sending their own tankers to get it _ said no to the Iraqi leader. Saddam's offer was seen as a none-too-subtle attempt to bypass the U.N. embargo, in effect since four days after Iraq's Aug. 2 invasion of Kuwait, by getting poor countries to dock their tankers in Iraq. But according to a State Department survey, Cuba and Romania have struck oil deals with Iraq and companies elsewhere are trying to continue trade with Baghdad, all in defiance of U.N. sanctions. Romania denies the allegation. The report, made available to The Associated Press, said some Eastern European countries also are trying to maintain their military sales to Iraq. A well-informed source in Tehran told The Associated Press that Iran has agreed to an Iraqi request to exchange food and medicine for up to 200,000 barrels of refined oil a day and cash payments. There was no official comment from Tehran or Baghdad on the reported food-for-oil deal. But the source, who requested anonymity, said the deal was struck during Iraqi Foreign Minister Tariq Aziz's visit Sunday to Tehran, the first by a senior Iraqi official since the 1980-88 gulf war. After the visit, the two countries announced they would resume diplomatic relations. Well-informed oil industry sources in the region, contacted by The AP, said that although Iran is a major oil exporter itself, it currently has to import about 150,000 barrels of refined oil a day for domestic use because of damages to refineries in the gulf war. Along similar lines, ABC News reported that following Aziz's visit, Iraq is apparently prepared to give Iran all the oil it wants to make up for the damage Iraq inflicted on Iran during their conflict. Secretary of State James A. Baker III, meanwhile, met in Moscow with Soviet Foreign Minister Eduard Shevardnadze, two days after the U.S.-Soviet summit that produced a joint demand that Iraq withdraw from Kuwait. During the summit, Bush encouraged Mikhail Gorbachev to withdraw 190 Soviet military specialists from Iraq, where they remain to fulfill contracts. Shevardnadze told the Soviet parliament Tuesday the specialists had not reneged on those contracts for fear it would jeopardize the 5,800,

Cracks appeared in the U.N. trade embargo against Iraq.

The State Department reports that Cuba and Romania have struck oil deals with Iraq as others attempt to trade with Baghdad in defiance of the sanctions.

Iran has agreed to exchange food and medicine for Iraqi oil.

Saddam has offered developing nations free oil if they send their tankers to pick it up.

ApproachApproach

Create a graph that represents a Create a graph that represents a semantic structure of the semantic structure of the documentdocument

Train a Train a machine learning modelmachine learning model for selecting sentences which for selecting sentences which includes the properties of the includes the properties of the semantic graph.semantic graph.

Cracks Appear in U.N. Trade Embargo Against Iraq.

Cracks appeared Tuesday in the U.N. trade embargo against Iraq as Saddam Hussein sought to circumvent the economic noose around his country. Japan, meanwhile, announced it would increase its aid to countries hardest hit by enforcing the sanctions. Hoping to defuse criticism that it is not doing its share to oppose Baghdad, Japan said up to $2 billion in aid may be sent to nations most affected by the U.N. embargo on Iraq. President Bush on Tuesday night promised a joint session of Congress and a nationwide radio and television audience that ``Saddam Hussein will fail'' to make his conquest of Kuwait permanent. ``America must stand up to aggression, and we will,'' said Bush, who added that the U.S. military may remain in the Saudi Arabian desert indefinitely. ``I cannot predict just how long it will take to convince Iraq to withdraw from Kuwait,'' Bush said. More than 150,000 U.S. troops have been sent to the Persian Gulf region to deter a possible Iraqi invasion of Saudi Arabia. Bush's aides said the president would follow his address to Congress with a televised message for the Iraqi people, declaring the world is united against their government's invasion of Kuwait. Saddam had offered Bush time on Iraqi TV. The Philippines and Namibia, the first of the developing nations to respond to an offer Monday by Saddam of free oil _ in exchange for sending their own tankers to get it _ said no to the Iraqi leader. Saddam's offer was seen as a none-too-subtle attempt to bypass the U.N. embargo, in effect since four days after Iraq's Aug. 2 invasion of Kuwait, by getting poor countries to dock their tankers in Iraq. But according to a State Department survey, Cuba and Romania have struck oil deals with Iraq and companies elsewhere are trying to continue trade with Baghdad, all in defiance of U.N. sanctions. Romania denies the allegation. The report, made available to The Associated Press, said some Eastern European countries also are trying to maintain their military sales to Iraq. A well-informed source in Tehran told The Associated Press that Iran has agreed to an Iraqi request to exchange food and medicine for up to 200,000 barrels of refined oil a day and cash payments. There was no official comment from Tehran or Baghdad on the reported food-for-oil deal. But the source, who requested anonymity, said the deal was struck during Iraqi Foreign Minister Tariq Aziz's visit Sunday to Tehran, the first by a senior Iraqi official since the 1980-88 gulf war. After the visit, the two countries announced they would resume diplomatic relations. Well-informed oil industry sources in the region, contacted by The AP, said that although Iran is a major oil exporter itself, it currently has to import about 150,000 barrels of refined oil a day for domestic use because of damages to refineries in the gulf war. Along similar lines, ABC News reported that following Aziz's visit, Iraq is apparently prepared to give Iran all the oil it wants to make up for the damage Iraq inflicted on Iran during their conflict. Secretary of State James A. Baker III, meanwhile, met in Moscow with Soviet Foreign Minister Eduard Shevardnadze, two days after the U.S.-Soviet summit that produced a joint demand that Iraq withdraw from Kuwait. During the summit, Bush encouraged Mikhail Gorbachev to withdraw 190 Soviet military specialists from Iraq, where they remain to fulfill contracts. Shevardnadze told the Soviet parliament Tuesday the specialists had not reneged on those contracts for fear it would jeopardize the 5,800,

Cracks appeared in the U.N. trade embargo against Iraq.

The State Department reports that Cuba and Romania have struck oil deals with Iraq as others attempt to trade with Baghdad in defiance of the sanctions.

Iran has agreed to exchange food and medicine for Iraqi oil.

Saddam has offered developing nations free oil if they send their tankers to pick it up.

Linguistic processing

and construction

of the semantic

graph

Sub-graph that characterizes extracted summaries

Sentence extraction based on the extracted sub- graph

Automatically generateddocument summary

Cracks Appear in U.N. Trade Embargo Against Iraq.

Cracks appeared Tuesday in the U.N. trade embargo against Iraq as Saddam Hussein sought to circumvent the economic noose around his country. Japan, meanwhile, announced it would increase its aid to countries hardest hit by enforcing the sanctions. Hoping to defuse criticism that it is not doing its share to oppose Baghdad, Japan said up to $2 billion in aid may be sent to nations most affected by the U.N. embargo on Iraq. President Bush on Tuesday night promised a joint session of Congress and a nationwide radio and television audience that ``Saddam Hussein will fail'' to make his conquest of Kuwait permanent. ``America must stand up to aggression, and we will,'' said Bush, who added that the U.S. military may remain in the Saudi Arabian desert indefinitely. ``I cannot predict just how long it will take to convince Iraq to withdraw from Kuwait,'' Bush said. More than 150,000 U.S. troops have been sent to the Persian Gulf region to deter a possible Iraqi invasion of Saudi Arabia. Bush's aides said the president would follow his address to Congress with a televised message for the Iraqi people, declaring the world is united against their government's invasion of Kuwait. Saddam had offered Bush time on Iraqi TV. The Philippines and Namibia, the first of the developing nations to respond to an offer Monday by Saddam of free oil _ in exchange for sending their own tankers to get it _ said no to the Iraqi leader. Saddam's offer was seen as a none-too-subtle attempt to bypass the U.N. embargo, in effect since four days after Iraq's Aug. 2 invasion of Kuwait, by getting poor countries to dock their tankers in Iraq. But according to a State Department survey, Cuba and Romania have struck oil deals with Iraq and companies elsewhere are trying to continue trade with Baghdad, all in defiance of U.N. sanctions. Romania denies the allegation. The report, made available to The Associated Press, said some Eastern European countries also are trying to maintain their military sales to Iraq. A well-informed source in Tehran told The Associated Press that Iran has agreed to an Iraqi request to exchange food and medicine for up to 200,000 barrels of refined oil a day and cash payments. There was no official comment from Tehran or Baghdad on the reported food-for-oil deal. But the source, who requested anonymity, said the deal was struck during Iraqi Foreign Minister Tariq Aziz's visit Sunday to Tehran, the first by a senior Iraqi official since the 1980-88 gulf war. After the visit, the two countries announced they would resume diplomatic relations. Well-informed oil industry sources in the region, contacted by The AP, said that although Iran is a major oil exporter itself, it currently has to import about 150,000 barrels of refined oil a day for domestic use because of damages to refineries in the gulf war. Along similar lines, ABC News reported that following Aziz's visit, Iraq is apparently prepared to give Iran all the oil it wants to make up for the damage Iraq inflicted on Iran during their conflict. Secretary of State James A. Baker III, meanwhile, met in Moscow with Soviet Foreign Minister Eduard Shevardnadze, two days after the U.S.-Soviet summit that produced a joint demand that Iraq withdraw from Kuwait. During the summit, Bush encouraged Mikhail Gorbachev to withdraw 190 Soviet military specialists from Iraq, where they remain to fulfill contracts. Shevardnadze told the Soviet parliament Tuesday the specialists had not reneged on those contracts for fear it would jeopardize the 5,800,

Original document

Cracks appeared in the U.N. trade embargo against Iraq.

The State Department reports that Cuba and Romania have struck oil deals with Iraq as others attempt to trade with Baghdad in defiance of the sanctions.

Iran has agreed to exchange food and medicine for Iraqi oil.

Saddam has offered developing nations free oil if they send their tankers to pick it up.

Semantic graph of the

original document

Learn the sub-graph selection model

Research QuestionsResearch Questions

How do the properties of the semantic How do the properties of the semantic graph influence summarization? graph influence summarization?

What are the important attributes for What are the important attributes for learning?learning?

What role does linguistic analysis play What role does linguistic analysis play in the summarization procedure?in the summarization procedure?

Can we relax the complexity of Can we relax the complexity of linguistic analysis?linguistic analysis?

Summarization Summarization Procedure Procedure

1.1. Split text into sentencesSplit text into sentences

2.2. Obtain logical form of each Obtain logical form of each sentence using Microsoft sentence using Microsoft NLPWin parserNLPWin parser

3.3. Extract and link named Extract and link named entitiesentities

– ’’George Bush’ linked with George Bush’ linked with ‘Bush’ and ‘President’‘Bush’ and ‘President’

4.4. Perform partial anaphora Perform partial anaphora resolution resolution

– Replace pronoun Replace pronoun references to the objects references to the objects by their name (for he, she, by their name (for he, she, they, …) they, …)

5.5. Extract Subject-Predicate-Extract Subject-Predicate-Object triples (SPO) Object triples (SPO)

6.6. Connect SPO triples from Connect SPO triples from sentences to form the sentences to form the semantic graph of the semantic graph of the document.document.

Tom went to town. In a bookstore he bought a large book.

Tom went to town. In a bookstore he [Tom] bought a large book.

Tom go townTom buy book

Summarization Summarization Procedure Procedure

1.1. Split text into sentencesSplit text into sentences

2.2. Obtain logical form of each Obtain logical form of each sentence using Microsoft sentence using Microsoft NLPWin parserNLPWin parser

3.3. Extract and link named Extract and link named entitiesentities

– ’’George Bush’ linked with George Bush’ linked with ‘Bush’ and ‘President’‘Bush’ and ‘President’

4.4. Perform partial anaphora Perform partial anaphora resolution resolution

– Replace pronoun Replace pronoun references to the objects references to the objects by their name (for he, she, by their name (for he, she, they, …) they, …)

5.5. Extract Subject-Predicate-Extract Subject-Predicate-Object triples (SPO) Object triples (SPO)

6.6. Connect SPO triples from Connect SPO triples from sentences to form the sentences to form the semantic graph of the semantic graph of the document.document.

Tom went to town. In a bookstore he bought a large book.

Tom went to town. In a bookstore he [Tom] bought a large book.

Tom go townTom buy book

Syntactic parse trees are used Syntactic parse trees are used to generate different linguistic to generate different linguistic representations sentencesrepresentations sentences

From Simple to Complex From Simple to Complex Linguistic AnalysisLinguistic Analysis

Parse tree

Logical form

– ANVANV: adjectives, nouns and verbs: adjectives, nouns and verbs– NPVNPV: head nouns and verbs: head nouns and verbs

– LFLF: logical form triples with syntactic and : logical form triples with syntactic and semantic tagssemantic tags

Example: “Jure sent Marko a letter”

Triples: Jure sent MarkoJure sent letter

NodesNodes representation: representation:– Each element is a node. Each element is a node. – Links connect elements Links connect elements

from the same triplefrom the same triple

Creation of the Semantic Creation of the Semantic GraphsGraphs

NamedLink NamedLink representation:representation:– Link is labeled with a verbLink is labeled with a verb

Thomas

standtake

Thomas

standtake

Text coverage by the graphText coverage by the graph

Relaxing linguistic processing increases the coverageRelaxing linguistic processing increases the coverage

Semantic GraphSemantic GraphStructureStructure

Covered Covered SummarySummary

sentences [%]sentences [%]

Covered non-Covered non-summarysummary

sentences [%]sentences [%]ANV with TriplesANV with Triples 93.493.4 86.686.6

ANV with Triples + PairsANV with Triples + Pairs 98.698.6 94.694.6

NPV with TriplesNPV with Triples 73.473.4 63.663.6

NPV with Triples + PairsNPV with Triples + Pairs 94.094.0 83.383.3

LF with TriplesLF with Triples 80.080.0 69.869.8

LF with Triples + PairsLF with Triples + Pairs 90.690.6 87.487.4

QUESTIONQUESTION

What is the impact of the document representation and What is the impact of the document representation and coverage on summarization performance?coverage on summarization performance?

Learning sub-structure of the Learning sub-structure of the graphgraph

LEARNING PROBLEM:LEARNING PROBLEM:

Given a semantic graph, learn to Given a semantic graph, learn to select parts that describe the select parts that describe the document summary.document summary.

Binary classification problem:Binary classification problem:– Positive examples are triples Positive examples are triples

from the human selected from the human selected sentences sentences

– Negative examples are all Negative examples are all other triplesother triples

Train the linear Support Vector Train the linear Support Vector classifier to detect ‘positive’ classifier to detect ‘positive’ triplestriples

To each triple from a document To each triple from a document graph the classifier assigns a graph the classifier assigns a confidence score that the triple confidence score that the triple belongs to a summary sentence. belongs to a summary sentence.

Label nodes and links that correspond to the summary sentences

Cracks Appear in U.N. Trade Embargo Against Iraq.

Cracks appeared Tuesday in the U.N. trade embargo against Iraq as Saddam Hussein sought to circumvent the economic noose around his country. Japan, meanwhile, announced it would increase its aid to countries hardest hit by enforcing the sanctions. Hoping to defuse criticism that it is not doing its share to oppose Baghdad, Japan said up to $2 billion in aid may be sent to nations most affected by the U.N. embargo on Iraq. President Bush on Tuesday night promised a joint session of Congress and a nationwide radio and television audience that ``Saddam Hussein will fail'' to make his conquest of Kuwait permanent. ``America must stand up to aggression, and we will,'' said Bush, who added that the U.S. military may remain in the Saudi Arabian desert indefinitely. ``I cannot predict just how long it will take to convince Iraq to withdraw from Kuwait,'' Bush said. More than 150,000 U.S. troops have been sent to the Persian Gulf region to deter a possible Iraqi invasion of Saudi Arabia. Bush's aides said the president would follow his address to Congress with a televised message for the Iraqi people, declaring the world is united against their government's invasion of Kuwait. Saddam had offered Bush time on Iraqi TV. The Philippines and Namibia, the first of the developing nations to respond to an offer Monday by Saddam of free oil _ in exchange for sending their own tankers to get it _ said no to the Iraqi leader. Saddam's offer was seen as a none-too-subtle attempt to bypass the U.N. embargo, in effect since four days after Iraq's Aug. 2 invasion of Kuwait, by getting poor countries to dock their tankers in Iraq. But according to a State Department survey, Cuba and Romania have struck oil deals with Iraq and companies elsewhere are trying to continue trade with Baghdad, all in defiance of U.N. sanctions. Romania denies the allegation. The report, made available to The Associated Press, said some Eastern European countries also are trying to maintain their military sales to Iraq. A well-informed source in Tehran told The Associated Press that Iran has agreed to an Iraqi request to exchange food and medicine for up to 200,000 barrels of refined oil a day and cash payments. There was no official comment from Tehran or Baghdad on the reported food-for-oil deal. But the source, who requested anonymity, said the deal was struck during Iraqi Foreign Minister Tariq Aziz's visit Sunday to Tehran, the first by a senior Iraqi official since the 1980-88 gulf war. After the visit, the two countries announced they would resume diplomatic relations. Well-informed oil industry sources in the region, contacted by The AP, said that although Iran is a major oil exporter itself, it currently has to import about 150,000 barrels of refined oil a day for domestic use because of damages to refineries in the gulf war. Along similar lines, ABC News reported that following Aziz's visit, Iraq is apparently prepared to give Iran all the oil it wants to make up for the damage Iraq inflicted on Iran during their conflict. Secretary of State James A. Baker III, meanwhile, met in Moscow with Soviet Foreign Minister Eduard Shevardnadze, two days after the U.S.-Soviet summit that produced a joint demand that Iraq withdraw from Kuwait. During the summit, Bush encouraged Mikhail Gorbachev to withdraw 190 Soviet military specialists from Iraq, where they remain to fulfill contracts. Shevardnadze told the Soviet parliament Tuesday the specialists had not reneged on those contracts for fear it would jeopardize the 5,800,

Learned summary graph (1):Learned summary graph (1):War in KuwaitWar in Kuwait

Nodes like “Saddam Hussein”, “President Bush”, “gulf force” are central in the graph

Learned summary graph (2)Learned summary graph (2)Russian presidential electionRussian presidential election

Learned summary graph (3):Learned summary graph (3):Bill Clinton giving a talkBill Clinton giving a talk

Attributes of triples used in Attributes of triples used in learning learning

Positional informationPositional information – Of the sentence from which the triple Of the sentence from which the triple

was derived relative to the document was derived relative to the document texttext

– Of the triple relative to the beginning of Of the triple relative to the beginning of the sentencethe sentence

NLPWin NLPWin linguistic attributeslinguistic attributes of the of the nodes in the triple:nodes in the triple:– 18 syntactic attributes18 syntactic attributes

– 100 semantic attributes100 semantic attributes

14 14 graph attributesgraph attributes: PageRank, In/Out : PageRank, In/Out Degree, reachable neighbours, etc. Degree, reachable neighbours, etc.

Extract and evaluate Extract and evaluate summariessummaries

SUMMARY EXTRACTION:SUMMARY EXTRACTION: Score each sentence in the Score each sentence in the

document as a sum of confidence document as a sum of confidence scores associated with triples from scores associated with triples from the sub-graph the sub-graph

Include into the summary a pre-Include into the summary a pre-defined number of high scoring defined number of high scoring sentences from the documentsentences from the document

EVALUATE SUMMARIESEVALUATE SUMMARIES Report microaveraged precision, Report microaveraged precision,

recall, and F1 on the sentence recall, and F1 on the sentence levellevel

Calculate ROUGE score to Calculate ROUGE score to measure the coverage of conceptsmeasure the coverage of concepts

Use node confidence

scores to select sentences

Cracks appeared in the U.N. trade embargo against Iraq.

The State Department reports that Cuba and Romania have struck oil deals with Iraq as others attempt to trade with Baghdad in defiance of the sanctions.

Iran has agreed to exchange food and medicine for Iraqi oil.

Saddam has offered developing nations free oil if they send their tankers to pick it up.

DUC2002 – Document Understanding DUC2002 – Document Understanding ConferenceConference– 300300 newspaper articles newspaper articles on on 30 30 different topics: different topics:

people, natural disasters, events, … people, natural disasters, events, … – Each topic has about 10 articlesEach topic has about 10 articles– 147 of these articles have document extracts147 of these articles have document extracts

CAST DataCAST Data– Subset of 89 documents with marked by a Subset of 89 documents with marked by a

single annotatorsingle annotator– 2 levels of granularity:2 levels of granularity:

CAST-15%: 6 sentences marked as essentialCAST-15%: 6 sentences marked as essential CAST-30%: additional 6 sentences marked as CAST-30%: additional 6 sentences marked as

importantimportant

DatasetsDatasets

Influence of Linguistic Influence of Linguistic AnalysisAnalysis

The Logical Form The Logical Form representation representation yields highest F1 yields highest F1 of extracted of extracted sentencessentences

Generally, SVM Generally, SVM learns well from all learns well from all levels of linguistic levels of linguistic processingprocessing

By ROUGE the By ROUGE the simplest linguistic simplest linguistic analysis, with analysis, with highest coverage, highest coverage, performs bestperforms best

DUC 2002DUC 2002

Linguistic UnitsLinguistic Units Prec.Prec. RecallRecall F1F1 RougeRouge

ANVANV 0.390.39 0.400.40 0.400.40 0.670.67NPVNPV 0.370.37 0.390.39 0.380.38 0.650.65LFLF 0.400.40 0.400.40 0.400.40 0.640.64

CAST-30%CAST-30%

Linguistic UnitsLinguistic Units Prec.Prec. RecallRecall F1F1 RougeRouge

ANVANV 0.430.43 0.570.57 0.490.49 0.670.67NPVNPV 0.410.41 0.660.66 0.500.50 0.630.63LFLF 0.420.42 0.670.67 0.520.52 0.660.66

Experiment set up:Experiment set up: Graph created using triples and pairs. Graph created using triples and pairs.

Learning based on all the attributesLearning based on all the attributes

Semantic Graph Semantic Graph representationrepresentation

Increasing the coverage of sentences by including node Increasing the coverage of sentences by including node pairs does not hurt the performancepairs does not hurt the performance

The ROUGE score slightly improvesThe ROUGE score slightly improves

DUC 2002DUC 2002

Semantic StructureSemantic Structure Prec.Prec. RecallRecall F1F1 RougeRouge

ANV with TriplesANV with Triples 0.310.31 0.620.62 0.410.41 0.670.67

ANV with Triples+PairsANV with Triples+Pairs 0.300.30 0.630.63 0.410.41 0.660.66

NPV with TriplesNPV with Triples 0.290.29 0.620.62 0.400.40 0.650.65

NPV with Triples+PairsNPV with Triples+Pairs 0.300.30 0.640.64 0.410.41 0.670.67

LF with TriplesLF with Triples 0.390.39 0.390.39 0.390.39 0.640.64

LF with Triples+PairsLF with Triples+Pairs 0.380.38 0.390.39 0.380.38 0.650.65

Influence of the attribute Influence of the attribute setssets For all levels of linguistic processing, the performance For all levels of linguistic processing, the performance

increases when we include graph attributesincreases when we include graph attributes

Node AttributesNode Attributes Prec.Prec. RecallRecall F1F1 RougeRouge

ANV: Pos + LingANV: Pos + Ling 0.270.27 0.580.58 0.370.37 0.650.65

ANV: Pos + Ling + ANV: Pos + Ling + GraphGraph 0.390.39 0.400.40 0.400.40 0.670.67

NPV: Pos + LingNPV: Pos + Ling 0.370.37 0.350.35 0.360.36 0.620.62

NPV: Pos + Ling + NPV: Pos + Ling + GraphGraph 0.370.37 0.390.39 0.380.38 0.650.65

LF: Pos + LingLF: Pos + Ling 0.260.26 0.610.61 0.360.36 0.620.62

LF: Pos + Ling + LF: Pos + Ling + GraphGraph 0.400.40 0.400.40 0.400.40 0.640.64

ConclusionConclusion

We investigated the impact of the We investigated the impact of the level level of linguistic analysesof linguistic analyses and and attributes of attributes of the semantic graphthe semantic graph on the SVM on the SVM classifier for summary extractionclassifier for summary extraction

Relaxing the structure of individual Relaxing the structure of individual sentence representation yields a wider sentence representation yields a wider text coveragetext coverage

Using graph attributes yields better Using graph attributes yields better summary performancesummary performance

Wider text coverage yields higher Wider text coverage yields higher ROUGE scoresROUGE scores