www.data61.csiro.au

HowtoWriteaGoodPaper

GernotHeiser|gernot.heiser@data61.csiro.au|@GernotHeiser

TSBootcampMay’18

...orwhatIwishmysupervisorhadtaughtme

TSBootcampMay'182|

RejectionIsLife

• My2013stats(mysecond-bestyearever!):–  11accepts:–  6conferences:EuroSys,SIGMOD,SOSP,OOPSLA,2*RTAS–  4workshops:HotOS,APSys,PLOS,HotPower,–  1journal:TOCS(plusTODSinvite)

–  8rejects:2*Usenix,PLDI,2*RTSS,APSys,EMSOFT,RTAS• My2017stats(abadyear):–  4accepts:–  1conference:EuroSys(paperrejected5times!)–  2workshops:PLOS,APSys–  1magazine(invited):IEEEDesign&Test

–  7rejects:–  UsenixSecurity,IEEES&P×2,RTAS,ASPLOS,SOSP×2

3| TSBootcampMay'18

Qualifications?

IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense4|

•  Servedonalltop-tierconferenceTPCsinmyfield•  atleastonetop-tierPCperyear•  Presently•  associateeditorofIEEETransactionsonComputers•  EBmemberofACMCommunicationsResearchHighlights

WaystoAvoidRejection

•  Safeway:Aimlow:•  2nd/3rd-tiervenuesareeasy•  guaranteedimpact-free

IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense5|

THIS ADVICE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

Onlywaytobuildreputation!

•  Hardway:Writeanexcellentpaperwithasignificantcontribution

JournalsvsConferences

•  I’minComputerScience,sub-disciplineOperatingSystems•  90%oftopOSpublicationsareinconferences•  othersub-disciplinesaredifferent:CStheoryisabout50%journals•  CStopconferencereviewingmuchtougherthanjournals•  Non-CSconferencesaremorelightweight,journalsdominate

IEEEAuthoringWorkshop,UNSW,July'18©GernotHeiser2018,releasedunderCCAttributionLicense6|

ComputerScienceEspeciallysystems

Other

Complete,significantwork

Top-tierconference Transactions

Lesssignificant,recycledrejects

Secondaryconference,journal

Otherjournal

Earlywork,positionpapers

Workshop Conference

Whatis“Systems”?

(Overly?)simplifiedviewofComputerscience:theory+systems•  Theoristsbuildtheories,models•  oftengetawaywiththeoriesnotgoodforanything•  Systemsfolksbuildstuff•  don’tgetawaywithworknotgoodforanything!

Examplesof“systems”work:•  operatingsystems•  networksystems/distributedsystems•  databasesystems•  programmingsystems(PLimplementation)• machine-learningsystems•  …

Mycommunity,i.e.Iunderstandhowit

ticks

TSBootcampMay'187|

RulesofWriting

Rule1:ReviewersarePotLuck

•  …evenattopconferences•  evengoodpapersgetrejected,sometimesforthewrongreasons• …butmosttimesrejectionisyourfault!

•  Reviewers’topreasonsforrejection•  I’mnotconvincedyou’resolvingarealproblem•  I’mnotconvincedyou’resolvingtheproblem•  Idon’tunderstand–yourpaperistoobadlywritten•  Insufficientcontributionfor{SOSP,OSDI,EuroSys…}

•  PaperswithoutaPC“champion”haveahardstand• Makesurethere’ssomethingwhichatleastonereviewerwillthinkcool•  Purelyincrementalworkwillhaveahardstandattopvenues

TSBootcampMay'189|

Rule2:APaperHasaStory

1.  Thepaperhasa(one!)mainmessage•  Understandclearlywhatthemessageis•  Makesurethatthereadergetsit•  Makesureit’saninterestingone

2.  Apaperhasanarrative•  Itstartsfromzeroandthenworksontransmittingthemessage•  Everythingyouwritemustsupportthemessage•  Maintainreaderstate!

–  beconsciousofwhatthereaderknows/remembers

TSBootcampMay'1810|

Rule3:LimitedSpace:TheTwo“C”s

•  Beclear(atalllevels)•  everysentence,paragraph,sectionhasaclearpurpose•  thepurposeisclearlycommunicated•  theoverallmessageisconsistent•  Beconcise(briefbutcomplete)•  don’twaffle!!!(Use“Jay’sruleofthumb”)•  beprecise• makesureit’sreadable,lucid,enjoyable

But:• maintainreaderstate:–  definebeforeuse–  beawareofwhatthereaderhaslearned–  recall/remindifnecessary

• Makesureit’sself-containedTSBootcampMay'1811|

Rule4:PresentationMatters

TSBootcampMay'1812|

Topconferencestendtoaccepttwokindsofpapers

1.  excellentworkthatispresentedwell

2.  averageworkthatispresentedwell

Thebestworkisuselessifyoucan’tconvincethereviewers•  reviewersarebusy,mayhavetoreview30papers•  they’lllookforreasonstoreject–don’tgivethemany!

Rule4:PresentationMatters–PaperEngineering

Importantbits:•  Introduction:selltheidea,thesignificanceandtheapproach

•  buildtension,makereaderinterested

•  convincingargumentation

•  top-down,notbottom-up

•  maintainreaderstate

•  convincingevaluation

•  thoroughandhonest

•  stateassumption/limitationshonestly

TSBootcampMay'1813|

PaperStructureandStyle

Introduction:MostImportantPartofthePaper!TheOverture:•  Explaintheproblemyou’resolving,whyit’saproblem•  Outlineyourapproach•  Indicateresults/outcomes•  Statecontributions•  “Paperroadmap”isawasteofspace

Generalhintsforintro:•  Capturethereader’sinterest:sellyouridea•  Beconcise:Staywithinaboutonepage!• Makesurethepaperdeliverswhatyoupromise•  Reviewerskillfor“baitandswitch”

TSBootcampMay'1815|

OtherParts

•  Background:setthesceneinmoredetail•  citerelatedworkasneeded,don’tdiscussmorethannecessary•  Examples!!!!•  Describeproblemindetail•  Explainsolutionindetail•  behonestandforthcomingwithlimitationsandassumptions•  Evaluation:forsystemsworkoftenlargestpart•  Relatedwork•  Conclusions•  Abstract•  usedtosteertotherightreviewers• What,Why,Achievement,Implication,onesentenceeach•  IMPORTANT:Redoforcamera-ready!

TSBootcampMay'1816|

Evaluation

•  Showthatyoursolutionactuallyworks•  Progressive:significantimprovementsinimportantsituations•  Conservative:no(significant)degradationelsewhereNeedboth!

TSBootcampMay'1817|

BenchmarkingCrimes(Selection)

1.  SelectiveBenchmarking–cherrypicking2.  Onlymicro-benchmarks3.  Throughputdegradation=overhead4.  Creativeoverheadaccounting5.  Improperbaseline,onlyrelativefigures,compareagainstself6.  Noindicationofsignificance

Fulllist:http://gernot-heiser.org/benchmarking-crimes.html

TSBootcampMay'1818|

StyleandForm• Writeinengagingstyle,leadreaderthoughthepaper•  avoidbottom-upstructure,presentideastop-down•  followstylerules•  Useactivevoice!!!!…andpresenttense•  Avoidbuzzwords(“novel”,“mobilesocialpost-quantumfogcomputing”)•  Bemindfulofreader’sbrainstate(whichislossy)• maintainreaderstate•  don’tassumeeveryreviewerisexpertinyournarrowarea•  butdon’tthinkyoucanhidestufffromreviewers!•  Followformattingrules•  don’tplaywithmargin,baselineskipetc•  don’tusemicroscopicfonts,>40yoldshaveproblemswith<8ptfont•  Spell-check,proof-read,proof-read•  getnativespeakertoproof-readifyouaren’t•  getoutsidertoreadit–greatwaytospotholesbeforeit’stoolate!TSBootcampMay'1819|

Mechanics•  Userevisioncontrol•  especially(butnotonly)whenit’sajointpaper

•  Don’tuseMSWord•  doesn’tintegratewellwithrevisioncontrol•  requirescoarse-grainlocking•  referencesarepainful,formulaeevenmoreso• MSRpeopleuseLaTeX,soshouldyou!

•  UseBibTeX•  …butuseitcorrectly(egcapitalisationintitles)

•  Usescriptabletools(egGNUplot)forgraphingresults•  Resultschangefrequentlyandatthelastminute•  Beingabletorunfromcommandlineisessential

TSBootcampMay'1820|

Summary

•  Clearstatementofproblem• WhywouldIcare?•  Convincingsolution,compellingargument•  Thoroughevaluation,noBMcrimes•  Lucidwriting,maintainingreaderstate

TSBootcampMay'1821|

FurtherReading

Writingsystemspapers:•  Levin&Redell:Anevaluationofthe9thSOSPsubmissions,orHow(andhownot)towriteagoodsystemspaper

•  SimonPeytonJones(MSRC):Howtowriteagreatresearchpaper•  http://research.microsoft.com/en-us/um/people/simonpj/papers/giving-a-talk/giving-a-talk-slides.pdf

•  Mypaper/thesiswritingguide•  http://gernot-heiser.org/style-guide.html

Generalwriting/styleetc(recommendedbysystemsfolks):•  Zobel:Writingforcomputerscience,Springer•  Strunk&White:Theelementsofstyle,Allyn&Bacon•  Dupré:Bugsinwriting:Aguidetodebuggingyourprose,Addison-Wesley

TSBootcampMay'1822|

www.csiro.au

Thankyou

TSBootcampMay'1823|