Lecture15:Summariza0on

AlanRi6er(many slides from Greg Durrett)

ThisLecture

‣ Extrac0vesystemsformul0-documentsummariza0on

‣ Extrac0ve+compressivesystemsforsingle-documentsummariza0on

‣ Single-documentsummariza0onwithneuralnetworks

Summariza0on

Summariza0on

Summariza0on

Summariza0on

Summariza0on

‣Whatmakesagoodsummary?

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onBAGHDAD/ERBIL,Iraq(Reuters)-AstrongearthquakehitlargepartsofnorthernIraqandthecapitalBaghdadonSunday,andalsocauseddamageinvillagesacrosstheborderinIranwherestateTVsaidatleastsixpeoplehadbeenkilled.

Therewerenoimmediatereportsofcasual0esinIraqaUerthequake,whoseepicenterwasinPenjwin,inSulaimaniyahprovincewhichisinthesemi-autonomousKurdistanregionveryclosetotheIranianborder,accordingtoanIraqimeteorologyofficial.

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

TheUSGeologicalSurveysaidthequakemeasuredamagnitudeof7.3,whileanIraqimeteorologyofficialputitsmagnitudeat6.5accordingtopreliminaryinforma0on.

ManyresidentsintheIraqicapitalBaghdadrushedoutofhousesandtallbuildingsinpanic.…

Summariza0onIndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.Electricityhasalsobeendisruptedatmanyplaces,suggestfewTVreports.

Summariza0onIndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.Electricityhasalsobeendisruptedatmanyplaces,suggestfewTVreports.

Summariza0onIndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.Electricityhasalsobeendisruptedatmanyplaces,suggestfewTVreports.

Amassiveearthquakeofmagnitude7.3struckIraqonSunday.TheepicenterwasclosetotheIranianborder.EightvillagesweredamagedandsixpeoplewerekilledinIran.

Summary

Summariza0onIndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.Electricityhasalsobeendisruptedatmanyplaces,suggestfewTVreports.

Amassiveearthquakeofmagnitude7.3struckIraqonSunday.TheepicenterwasclosetotheIranianborder.EightvillagesweredamagedandsixpeoplewerekilledinIran.

Summary

Whatmakesagoodsummary?

Astrongearthquakeofmagnitude7.3struckIraqandIranonSunday.TheepicenterwasclosetotheIranianborder.EightvillagesweredamagedandsixpeoplewerekilledinIran.

Summary

Whatmakesagoodsummary?

Astrongearthquakeofmagnitude7.3struckIraqandIranonSunday.TheepicenterwasclosetotheIranianborder.EightvillagesweredamagedandsixpeoplewerekilledinIran.

Summary

‣ Contentselec0on:picktherightcontent‣ Rightcontentwasrepeatedwithinandacrossdocuments

‣ Domain-specific(magnitude+epicenterofearthquakesareimportant)

Whatmakesagoodsummary?

Astrongearthquakeofmagnitude7.3struckIraqandIranonSunday.TheepicenterwasclosetotheIranianborder.EightvillagesweredamagedandsixpeoplewerekilledinIran.

Summary

‣ Contentselec0on:picktherightcontent‣ Rightcontentwasrepeatedwithinandacrossdocuments

‣ Domain-specific(magnitude+epicenterofearthquakesareimportant)

‣ Genera0on:writethesummary‣ Extrac0on:pickwholesentencesfromthesummary

‣ Compression:compressthosesentencesbutbasicallyjustdodele0on‣ Abstrac0on:rewrite+reexpresscontentfreely

Extrac0veSummariza0on

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

Calculate

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

Calculate

“maxoverallsentencesnotyetinthesummary”

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

Calculate

“makethissentencesimilartoaquery”

“maxoverallsentencesnotyetinthesummary”

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

Calculate

“makethissentencesimilartoaquery”

“makethissentencemaximallydifferentfromallothersaddedsofar”

“maxoverallsentencesnotyetinthesummary”

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:MMR‣ Givensomear0clesandalengthbudgetofkwords,picksomesentencesoftotallength<=kandmakeasummary

‣ Pickimportantyetdiversecontent:maximummarginalrelevance(MMR)

Whilesummaryis<kwords

Calculate

AddhighestMMRsentencethatdoesn’toverflowlength

“makethissentencesimilartoaquery”

“makethissentencemaximallydifferentfromallothersaddedsofar”

“maxoverallsentencesnotyetinthesummary”

CarbonellandGoldstein(1998)

Extrac0veSummariza0on:Centroid‣ Representthedocumentsandeachsentencesasbag-of-wordswithTF-IDFweigh0ng

Radevetal.(2004)

Extrac0veSummariza0on:Centroid‣ Representthedocumentsandeachsentencesasbag-of-wordswithTF-IDFweigh0ng

Whilesummaryis<kwords

Radevetal.(2004)

Extrac0veSummariza0on:Centroid‣ Representthedocumentsandeachsentencesasbag-of-wordswithTF-IDFweigh0ng

Whilesummaryis<kwords

Calculatescore(sentence)=cosine(sent-vec,doc-vec)

Radevetal.(2004)

Extrac0veSummariza0on:Centroid‣ Representthedocumentsandeachsentencesasbag-of-wordswithTF-IDFweigh0ng

Whilesummaryis<kwords

Calculatescore(sentence)=cosine(sent-vec,doc-vec)

Radevetal.(2004)

Discardallsentenceswhosesimilaritywithsomesentencealreadyinthesummaryistoohigh

Extrac0veSummariza0on:Centroid‣ Representthedocumentsandeachsentencesasbag-of-wordswithTF-IDFweigh0ng

Whilesummaryis<kwords

Calculatescore(sentence)=cosine(sent-vec,doc-vec)

Radevetal.(2004)

Discardallsentenceswhosesimilaritywithsomesentencealreadyinthesummaryistoohigh

Addthebestremainingsentencethatwon’toverflowthesummary

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

score(massiveearthquake)=3

score(Iraqicapital)=1score(sixkilled)=2score(magnitude7.3)=2

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

score(massiveearthquake)=3

score(Iraqicapital)=1score(sixkilled)=2score(magnitude7.3)=2

‣ Findsummarythatmaximizesthescoreofbigramsitcovers

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

score(massiveearthquake)=3

score(Iraqicapital)=1score(sixkilled)=2score(magnitude7.3)=2

‣ ILPformula0on:candsareindicatorvariablesindexedoverbigrams(“concepts”)andsentences,respec0vely

‣ Findsummarythatmaximizesthescoreofbigramsitcovers

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

score(massiveearthquake)=3

score(Iraqicapital)=1score(sixkilled)=2score(magnitude7.3)=2

‣ ILPformula0on:candsareindicatorvariablesindexedoverbigrams(“concepts”)andsentences,respec0vely

sumofincludedsentences’lengthscan’texceedL

‣ Findsummarythatmaximizesthescoreofbigramsitcovers

Extrac0veSummariza0on:BigramRecall‣ Countnumberofdocumentseachbigramoccursintomeasureimportance

GillickandFavre(2009)

score(massiveearthquake)=3

score(Iraqicapital)=1score(sixkilled)=2score(magnitude7.3)=2

‣ ILPformula0on:candsareindicatorvariablesindexedoverbigrams(“concepts”)andsentences,respec0vely

“setcito1iffsomesentencethatcontainsitisincluded”

sumofincludedsentences’lengthscan’texceedL

‣ Findsummarythatmaximizesthescoreofbigramsitcovers

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

reference

predic0on

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

reference

predic0on

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

reference

predic0on

ROUGE2recall=1correctbigram(Iraq,Sunday)/3referencebigrams

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

reference

predic0on

ROUGE2recall=1correctbigram(Iraq,Sunday)/3referencebigrams

ROUGE2precision=1correctbigram(Iraq,Sunday)/6predictedbigrams

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

‣Manyhyperparameters:stemming,removestopwords,etc.

reference

predic0on

ROUGE2recall=1correctbigram(Iraq,Sunday)/3referencebigrams

ROUGE2precision=1correctbigram(Iraq,Sunday)/6predictedbigrams

Evalua0on:ROUGE‣ Rouge-n:n-gramprecision/recall/F1ofsummaryw.r.t.goldstandard

Lin(2004)

‣ Rouge-2correlateswellwithhumanjudgmentsformul0-documentsummariza0ontasks

AnearthquakewasdetectedinIraqonSunday

Amassiveearthquakeofmagnitude7.3struckIraqonSunday

‣Manyhyperparameters:stemming,removestopwords,etc.

reference

predic0on

ROUGE2recall=1correctbigram(Iraq,Sunday)/3referencebigrams

‣ Historically:ROUGErecall@k{words,characters}.Now:ROUGEF1

ROUGE2precision=1correctbigram(Iraq,Sunday)/6predictedbigrams

Results

Ghalandri(2017)

Be6ercentroid:38.589.731.53

GillickandFavre/bigramrecall

Results

Ghalandri(2017)

Be6ercentroid:38.589.731.53

GillickandFavre/bigramrecall

‣ Caveat:thesetechniquesallworkbe6erformul0-documentthansingle-document!

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

‣Whenyouhavealotofdocuments,therearemorepossiblesentencestoextract:

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

‣Whenyouhavealotofdocuments,therearemorepossiblesentencestoextract:

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

‣Whenyouhavealotofdocuments,therearemorepossiblesentencestoextract:

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

‣Whenyouhavealotofdocuments,therearemorepossiblesentencestoextract:

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.

Mul0-Documentvs.SingleDocument

‣ “amassiveearthquakehitIraq”“amassiveearthquakestruckIraq”—lotsofredundancytohelpselectcontentinmul0-documentcase

‣Whenyouhavealotofdocuments,therearemorepossiblesentencestoextract:

ButeightvillagesweredamagedinIranandatleastsixpeoplewerekilledandmanyothersinjuredinthebordertownofQasr-eShirininIran,IranianstateTVsaid.

ThequakehasbeenfeltinseveralIranianci0esandeightvillageshavebeendamaged.

‣Mul0-documentsummariza0oniseasier?

CompressiveSummariza0on

CompressiveSummariza0on

IndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

CompressiveSummariza0on

IndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

CompressiveSummariza0on

IndianExpress—Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)southeastofthecityofAs-Sulaymaniyah,theUSGeologicalSurveysaid,reportsReuters.USGeologicalSurveyini0allysaidthequakewasofamagnitude7.2,beforerevisingitto7.3.

‣ Sentenceextrac0onisn’taggressiveenoughatremovingirrelevantcontent

‣Wanttoextractsentencesandalsodeletecontentfromthem

Syntac0cCuts

‣ Deleteadjuncts

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

Syntac0cCuts

‣ Deleteadjuncts

Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)…

NP

NP-LOCVBD NP NP-TMP

VP

S

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

Syntac0cCuts

‣ Deleteadjuncts

Amassiveearthquakeofmagnitude7.3struckIraqonSunday,103kms(64miles)…

NP

NP-LOCVBD NP NP-TMP

VP

S

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

Syntac0cCuts

‣ Deletesecondpartsofcoordina0onstructures

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

Syntac0cCuts

‣ Deletesecondpartsofcoordina0onstructures

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

Syntac0cCuts

‣ Deletesecondpartsofcoordina0onstructures

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

‣ Usesyntac0crulestomakecertaindele0ons

Berg-Kirkpatricketal.(2011)

CompressiveILP‣ RecalltheGillick+FavreILP:

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

Berg-Kirkpatricketal.(2011)

CompressiveILP‣ RecalltheGillick+FavreILP:

‣ Nowsjvariablesarenodesorsetsofnodesintheparsetree

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

Berg-Kirkpatricketal.(2011)

CompressiveILP‣ RecalltheGillick+FavreILP:

‣ Nowsjvariablesarenodesorsetsofnodesintheparsetree

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

s2s1

Berg-Kirkpatricketal.(2011)

CompressiveILP‣ RecalltheGillick+FavreILP:

‣ Nowsjvariablesarenodesorsetsofnodesintheparsetree

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

s2s1

‣ Newconstraint:s2≤s1

Berg-Kirkpatricketal.(2011)

CompressiveILP‣ RecalltheGillick+FavreILP:

‣ Nowsjvariablesarenodesorsetsofnodesintheparsetree

Atleastsixpeoplewerekilledandmanyothersinjured

S SCC

S

s2s1

‣ Newconstraint:s2≤s1“s1isaprerequisitefors2”

Berg-Kirkpatricketal.(2011)

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

Theoat-brancrazehascostitmarketshare.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Theoat-brancrazehascostKelloggmarketshare.

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Theoat-brancrazehascostKelloggmarketshare.

x1

x2

x3

x4x5

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Theoat-brancrazehascostKelloggmarketshare.

x1

x2

x3

x4x5

maxx

�w>f(x)

�ILP:

Thishasn'tbeenKellogg'syear.

CompressiveSummariza0on

Itspresidentquitsuddenly.

AndnowKelloggiscancelingitsnewcerealplant,whichwouldhavecost$1billion.

SBARNP

NP

Theoat-brancrazehascostKelloggmarketshare.

x1

x2

x3

x4x5

maxx

�w>f(x)

�s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

ILP:

Constraints

maxx

�w>f(x)

�s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Durre6etal.(2016)

Constraints

maxx

�w>f(x)

�

Gramma0calityconstraints:allowcutswithinsentences

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Durre6etal.(2016)

Constraints

maxx

�w>f(x)

�

Gramma0calityconstraints:allowcutswithinsentences

Coreferenceconstraints:donotallowpronounsthatwouldrefertonothing

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Durre6etal.(2016)

Constraints

maxx

�w>f(x)

�

Gramma0calityconstraints:allowcutswithinsentences

Coreferenceconstraints:donotallowpronounsthatwouldrefertonothing

‣ Ifwe’reconfidentaboutcoreference,rewritethepronoun(itKellogg)

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Durre6etal.(2016)

Constraints

maxx

�w>f(x)

�

Gramma0calityconstraints:allowcutswithinsentences

Coreferenceconstraints:donotallowpronounsthatwouldrefertonothing

‣ Otherwise,forceitsantecedenttobeincludedinthesummary‣ Ifwe’reconfidentaboutcoreference,rewritethepronoun(itKellogg)

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Durre6etal.(2016)

Features

maxx

�w>f(x)

�s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Features

maxx

�w>f(x)

�

AndnowKelloggiscancelingitsnewcerealplantf ( )=

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Features

maxx

�w>f(x)

�

AndnowKelloggiscancelingitsnewcerealplantf ( )=

}s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Features

maxx

�w>f(x)

�

AndnowKelloggiscancelingitsnewcerealplantf ( )=

(NumContentWords=4)I}s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Centrality:‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Features

maxx

�w>f(x)

�

AndnowKelloggiscancelingitsnewcerealplantf ( )=(SentenceIndex=4)

(NumContentWords=4)I

I

}s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Documentposi0on:

Centrality:‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Features

maxx

�w>f(x)

�

AndnowKelloggiscancelingitsnewcerealplantf ( )=(SentenceIndex=4)

(FirstWord=And)

(NumContentWords=4)I

I

I

}s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Documentposi0on:

Lexicalfeatures:

Centrality:‣ Nowusesafeature-basedmodel,wherefeaturesiden0fygoodcontent

Learning

maxx

�w>f(x)

�s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Berg-Kirkpatricketal.(2011),Durre6etal.(2016)

Learning

maxx

�w>f(x)

�

‣ TrainonalargecorpusofNewYorkTimesdocumentswithsummaries(100,000documents)

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Berg-Kirkpatricketal.(2011),Durre6etal.(2016)

Learning

maxx

�w>f(x)

�

‣ StructuredSVMwithROUGEaslossfunc0on

‣ TrainonalargecorpusofNewYorkTimesdocumentswithsummaries(100,000documents)

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

Berg-Kirkpatricketal.(2011),Durre6etal.(2016)

Learning

maxx

�w>f(x)

�

‣ StructuredSVMwithROUGEaslossfunc0on

‣ TrainonalargecorpusofNewYorkTimesdocumentswithsummaries(100,000documents)

s.t. summary(x) obeys length limit

summary(x) is grammatical

summary(x) is coherent

‣ AugmenttheILPtokeeptrackofwhichbigramsareincludedornot,usetheseforloss-augmenteddecode

Berg-Kirkpatricketal.(2011),Durre6etal.(2016)

Results:NewYorkTimesCorpus

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference)

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference) Firstnwords

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference) Firstnwords

Yoshidaetal.(2014)

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference) Firstnwords

Extractsentences

Yoshidaetal.(2014)

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference) Firstnwords

Extractsentences

Extractclauses

Yoshidaetal.(2014)

30

35

40

45

50

5 6 7 8 9 10

Results:NewYorkTimesCorpus

Linguis0cQuality(HumanstudyonMechanicalTurk)

Contentfidelity(ROUGE-1:wordoverlapwithreference) Firstnwords

Extractsentences

Extractclauses Fullsystem

Yoshidaetal.(2014)

WhereWeareAt

‣ Extrac0vesystemsformul0-documentsummariza0on

‣ Extrac0ve+compressivesystemsforsingle-documentsummariza0on

‣ Single-documentsummariza0onwithneuralnetworks

WhereWeareAt

‣ Extrac0vesystemsformul0-documentsummariza0on

‣ Extrac0ve+compressivesystemsforsingle-documentsummariza0on

‣ Single-documentsummariza0onwithneuralnetworks

NeuralSummariza0on

Seq2seqSummariza0on

‣ Extrac0veparadigmisn’tallthatflexible,evenwithcompression

Chopraetal.(2016)

Seq2seqSummariza0on

‣ Extrac0veparadigmisn’tallthatflexible,evenwithcompression

‣ Trainingishard!ILPsarehard!Maybejustuseseq2seq?

Chopraetal.(2016)

Seq2seqSummariza0on

‣ Extrac0veparadigmisn’tallthatflexible,evenwithcompression

‣ Trainingishard!ILPsarehard!Maybejustuseseq2seq?

Itspresidentquitsuddenly

The

<s>

oat bran craze has

… …

Chopraetal.(2016)

‣ Traintoproducesummarybasedondocument

Seq2seqSummariza0on

Chopraetal.(2016)

‣ Task:generateheadlinefromfirstsentenceofar0cle(cangetlotsofdata!)

Seq2seqSummariza0on

Chopraetal.(2016)

‣ Task:generateheadlinefromfirstsentenceofar0cle(cangetlotsofdata!)

noa6en0onwitha6en0on

reference

sentence

Seq2seqSummariza0on

Chopraetal.(2016)

‣ Task:generateheadlinefromfirstsentenceofar0cle(cangetlotsofdata!)

noa6en0onwitha6en0on

reference

sentence

‣Workspre6ywell,thoughthesemodelscangenerateincorrectsummaries(whohasthekneeinjury?)

Seq2seqSummariza0on

Chopraetal.(2016)

‣ Task:generateheadlinefromfirstsentenceofar0cle(cangetlotsofdata!)

noa6en0onwitha6en0on

reference

sentence

‣Workspre6ywell,thoughthesemodelscangenerateincorrectsummaries(whohasthekneeinjury?)

‣Whathappensifwetrythisonalongerar0cle?

Seq2seqSummariza0on

Seeetal.(2017)

‣What’swrongwiththissummary?

Pointer-GeneratorModel

Seeetal.(2017)

‣ CopyingapproachlikeinJia+Liang

Seq2seqSummariza0on

Seeetal.(2017)

‣ Solu0ons:copymechanism,coverage,justlikeinMT…

‣ Thingsmights0llgowrong,nowayofpreven0ngthis…

NeuralAbstrac0veSystems

Seeetal.(2017)

‣ Howabstrac0veisthis,anyway?

NeuralExtrac0veSystems

Nallapa0etal.(2017)

pooling

NeuralSystems:Results

Seeetal.(2017)

‣ Abstrac0vesystemsdon’t/barelybeata“lead”baselineonROUGE

‣ Copymechanismandcoveragehelpsubstan0ally

ChallengesofSummariza0on

‣ Trueabstrac0on?

summaryar0cle extrac0on

abstrac0on

compressionsyntax

seman0cs??? seman0cs???

syntax

ChallengesofSummariza0on

‣ Trueabstrac0on?‣ Notreallynecessaryforar0cles

summaryar0cle extrac0on

abstrac0on

compressionsyntax

seman0cs??? seman0cs???

syntax

ChallengesofSummariza0on

‣ Trueabstrac0on?‣ Notreallynecessaryforar0cles‣ Genera0ngfromstructuredinforma0oncanusuallybedonewithtemplates…

summaryar0cle extrac0on

abstrac0on

compressionsyntax

seman0cs??? seman0cs???

syntax

Takeaways

‣ Extrac0vesystemsbuiltonheuris0cs/ILPsworkpre6ywell

‣ Compressioncanmakethingsbe6er,especiallyinthesingle-documentse}ng

‣ Neuralsystems(likeMTmodels)candoabstrac0vesummariza0on,buttheyoUenjustcopyinputs(ordeviatefrominputsinbadways)