1.1

1.2

1.2.1

1.2.2

1.2.3

1.2.4

1.2.5

1.2.6

1.3

1.3.1

1.3.2

1.3.3

1.4

1.4.1

1.4.2

1.4.3

1.4.4

1.4.5

1.4.6

1.4.7

1.4.8

1.4.9

1.5

1.5.1

1.5.2

1.5.3

1.5.4

1.5.5

1.5.6

1.5.7

1.6

1.6.1

1.6.2

1.6.3

1.6.4

1.6.5

1.6.6

1.7

1.8

TableofContentsIntroduction

WhyQUIC

RememberHTTP/2

TCPheadoflineblocking

TCPorUDP

Ossification

Secure

Reducedlatency

Process

IETF

ExperiencefromHTTP/2

Status

Protocolfeatures

UDP

Reliable

Streams

InOrder

Fasthandshakes

TLS1.3

Transportandapplication

HTTP/3overQUIC

Non-HTTPoverQUIC

HowQUICworks

Connections

ConnectionsuseTLS

Streams

0-RTT

SpinBit

Userspace

API

HTTP/3

HTTPS://URLs

BootstrapwithAlt-svc

QUICstreamsandHTTP/3

Prioritization

Serverpush

ComparisonwithHTTP/2

Commoncriticism

Thespecifications

2

1.9QUICv2

3

HTTP/3explainedThisbookeffortwasstartedinMarch2018.TheplanistodocumentHTTP/3anditsunderlyingprotocol:QUIC.Why,howtheywork,protocoldetails,theimplementationsandmore.

Thebookisentirelyfreeandismeanttobeacollaborativeeffortinvolvinganyoneandeveryonewhowantstohelpout.

PrerequisitesAreaderofthisbookispresumedtohaveabasicunderstandingofTCP/IPnetworking,thefundamentalsofHTTPandtheweb.ForfurtherinsightsandspecificsaboutHTTP/2werecommendfirstreadingupthedetailsinhttp2explained.

AuthorThisbookiscreatedandtheworkisstartedbyDanielStenberg.Danielisthefounderandleaddeveloperofcurl,theworld'smostwidelyusedHTTPclientsoftware.DanielhasworkedwithandonHTTPandinternetprotocolsforovertwodecadesandistheauthorofhttp2explained.

HomeThehomepageforthisbookisfoundatdaniel.haxx.se/http3-explained.

HelpoutIfyoufindmistakes,omissions,errorsorblatantliesinthisdocument,pleasesendusarefreshedversionoftheaffectedparagraphandwewillmakeamendedversions.Wewillgivepropercreditstoeveryonewhohelpsout.Ihopetomakethisdocumentbetterovertime.

Preferably,yousubmiterrorsorpullrequestsonthebook'sgithubpage.

LicenseThisdocumentandallitscontentsarelicensedundertheCreativeCommonsAttribution4.0license.

Introduction

4

WhyQUICQUICisaname,notanacronym.ItispronouncedexactlyliketheEnglishword"quick".

QUICisinmanywayswhatcouldbeseenasawayofdoinganewreliableandsecuretransportprotocolthatissuitableforaprotocollikeHTTPandthatcanaddresssomeoftheknownshortcomingsofdoingHTTP/2overTCPandTLS.Thelogicalnextstepinthewebtransportevolution.

QUICisnotlimitedtojusttransportingHTTP.Thedesiretomakethewebanddataingeneraldeliveredfastertoendusersisprobablythelargestreasonandpushthatinitiallytriggeredthecreationofthisnewtransportprotocol.

SowhycreateanewtransportprotocolandwhydoitontopofUDP?

WhyQUIC

5

RememberHTTP/2?TheHTTP/2specificationRFC7540waspublishedinMay2015andtheprotocolhassincethenbeenimplementedanddeployedwidelyacrosstheInternetandtheWorldWideWeb.

Inearly2018,almost40%ofthetop-1000websitesrunHTTP/2,around70%ofallHTTPSrequestsFirefoxissuesgetHTTP/2responsesbackandallmajorbrowsers,serversandproxiessupportit.

HTTP/2addressesawholeslewofshortcomingsinHTTP/1andwiththeintroductionofthesecondversionofHTTPuserscanstopusingabunchofwork-arounds.Someofwhichareprettyburdensomeonwebdevelopers.

OneoftheprimaryfeaturesofHTTP/2isthatitmakesuseofmultiplexing,sothatmanylogicalstreamsaresentoverthesamephysicalTCPconnection.Thismakesalotofthingsbetterandfaster.Itmakescongestioncontrolworkmuchbetter,itletsusersuseTCPmuchbetterandthusproperlysaturatethebandwidth,makestheTCPconnectionsmorelong-lived-whichisgoodsothattheygetuptofullspeedmorefrequentlythanbefore.Headercompressionmakesituselessbandwidth.

WithHTTP/2,browserstypicallyuseoneTCPconnectiontoeachhostinsteadoftheprevioussix.Infact,connectioncoalescingand"desharding"techniquesusedwithHTTP/2mayactuallyevenreducethenumberofconnectionsmuchmorethanso.

HTTP/2fixedtheHTTPheadoflineblockingproblem,whereclientshadtowaitforthefirstrequestinlinetofinishbeforethenextonecouldgoout.

RememberHTTP/2

6

TCPheadoflineblockingHTTP/2isdoneoverTCPandwithmuchfewerTCPconnectionsthanwhenusingearlierHTTPversions.TCPisaprotocolforreliabletransfersandyoucanbasicallythinkofitasanimaginarychainbetweentwomachines.Whatisbeingputoutonthenetworkinoneendwillendupintheotherend,inthesameorder-eventually.(Ortheconnectionbreaks.)

WithHTTP/2,typicalbrowsersdotensorhundredsofparalleltransfersoverasingleTCPconnection.

Ifasinglepacketisdropped,orlostinthenetworksomewherebetweentwoendpointsthatspeakHTTP/2,itmeanstheentireTCPconnectionisbroughttoahaltwhilethelostpacketisre-transmittedandfindsitswaytothedestination.SinceTCPisthis"chain",itmeansthatifonelinkissuddenlymissing,everythingthatwouldcomeafterthelostlinkneedstowait.

Anillustrationusingthechainmetaphorwhensendingtwostreamsoverthisconnection.Aredstreamandagreenstream:

ItbecomesaTCP-basedheadoflineblock!

Asthepacketlossrateincreases,HTTP/2performslessandlesswell.At2%packetloss(whichisaterriblenetworkquality,mindyou),testshaveproventhatHTTP/1usersareusuallybetteroff-becausetheytypicallyhaveuptosixTCPconnectionstodistributelostpacketsover.Thismeansforeverylostpackettheotherconnectionscanstillcontinue.

Fixingthisissueisnoteasy,ifatallpossible,withTCP.

IndependentstreamsavoidstheblockWithQUICthereisstillaconnectionsetupbetweenthetwoend-pointsthatmakestheconnectionsecureandthedatadeliveryreliable.

Whensettinguptwodifferentstreamsoverthisconnection,theyaretreatedindependentlysothatifanylinkgoesmissingforoneofthestreams,onlythatstream,thatparticularchain,hastopauseandwaitforthemissinglinktogetretransmitted.

Illustratedherewithoneyellowandonebluestreamsentbetweentwoend-points.

TCPheadoflineblocking

7

TCPheadoflineblocking

8

TCPorUDPIfwecan'tfixhead-of-lineblockingwithinTCP,thenintheoryweshouldbeabletomakeanewtransportprotocolnexttoUDPandTCPinthenetworkstack.OrperhapsevenuseSCTPwhichisatransportprotocolstandardizedbytheIETFinRFC4960withseveralofthedesiredcharacteristics.

However,inrecentyearseffortstocreatenewtransportprotocolshavealmostentirelybeenhaltedbecauseofthedifficultiesindeployingthemontheInternet.Deploymentofnewprotocolsishamperedbymanyfirewalls,NATs,routersandothermiddle-boxesthatonlyallowTCPorUDParedeployedbetweenusersandtheserverstheyneedtoreach.IntroducinganothertransportprotocolmakesN%oftheconnectionsfailbecausetheyarebeingblockedbyboxesthatseeitnotbeingUDPorTCPandthusevilorwrongsomehow.TheN%failurerateisoftendeemedtoohightobeworththeeffort.

Additionally,changingthingsinthetransportprotocollayerofthenetworkstacktypicallymeansprotocolsimplementedbyoperatingsystemkernels.Updatinganddeployingnewoperatingsystemkernelsisaslowprocessthatrequiressignificanteffort.ManyTCPimprovementsstandardizedbytheIETFarenotwidelydeployedorusedbecausetheyarenotbroadlysupported.

WhynotSCTP-over-UDPSCTPisareliabletransportprotocolwithstreams,andforWebRTCthereareevenexistingimplementationsusingitoverUDP.

ThiswasnotdeemedgoodenoughasaQUICalternativeduetoseveralreasons,including:

SCTPdoesnotfixthehead-of-line-blockingproblemforstreamsSCTPrequiresthenumberofstreamstobedecidedatconnectionsetupSCTPdoesnothaveasolidTLS/securitystorySCTPhasa4-wayhandshake,QUICoffers0-RTTQUICisabytestreamlikeTCP,SCTPismessage-basedQUICconnectionscanmigratebetweenIPaddressesbutSCTPcannot

Formoredetailsonthedifferences,seeAComparisonbetweenSCTPandQUIC.

TCPorUDP

9

OssificationTheinternetisanetworkofnetworks.ThereisequipmentsetupontheInternetinmanydifferentplacesalongthewaytomakesurethisnetworkofnetworksworksasitissupposedto.Thesedevices,theboxesthataredistributedoutinthenetwork,arewhatwesometimesrefertoasmiddle-boxes.Boxesthatsitsomewherebetweenthetwoend-pointsthataretheprimarypartiesinvolvedinatraditionalnetworkdatatransfer.

TheseboxesservemanydifferentspecificpurposesbutIthinkwecansaythatuniversallytheyareputtherebecausesomeonethinkstheymustbetheretomakethingswork.

Middle-boxesrouteIPpacketsbetweennetworks,theyblockmalicioustraffic,theydoNAT(NetworkAddressTranslation),theyimproveperformance,sometrytospyonthepassingtrafficandmore.

Inordertoperformtheirdutiestheseboxesmustknowaboutnetworkingandtheprotocolsthataremonitoredormodifiedbythem.Theyrunsoftwareforthispurpose.Softwarethatisnotalwaysupgradedfrequently.

WhiletheyaregluecomponentsthatkeeptheInternettogethertheyarealsooftennotkeepingupwiththelatesttechnology.Themiddleofthenetworktypicallydoesnotmoveasfastastheedges,astheclientsandtheserversoftheworld.

Thenetworkprotocolsthattheseboxesmightwanttoinspect,andhaveideasaboutwhatisokayandwhatisnotthenhavethisproblem:theseboxesweredeployedsometimeagowhentheprotocolshadafeaturesetofthattime.Introducingnewfeaturesorchangesinbehaviorthatwerenotknownbeforerisksendingupconsideredbadorillegalbysuchboxes.Suchtrafficmaywelljustbedroppedordelayedtothedegreethatusersreallydonotwanttousethosefeatures.

Thisiscalled"protocolossification".

ChangestoTCPalsosufferfromossification:someboxesbetweenaclientandtheremoteserverwillspotunknownnewTCPoptionsandblocksuchconnectionssincetheydonotknowwhattheoptionsare.Ifallowedtodetectprotocoldetails,systemslearnhowprotocolstypicallybehaveandovertimeitbecomesimpossibletochangethem.

Theonlytrulyeffectivewayto"combat"ossificationistoencryptasmuchofthecommunicationaspossibleinordertopreventmiddle-boxesfromseeingmuchoftheprotocolpassingthrough.

Ossification

10

SecureQUICisalwayssecure.Thereisnoclear-textversionoftheprotocolsotonegotiateaQUICconnectionmeansdoingcryptographyandsecuritywithTLS1.3.Asmentionedabove,thispreventsossificationaswellasothersortsofblocksandspecialtreatments,aswellasmakingsureQUIChasallthesecurepropertiesofHTTPSthatwebusershavecometoexpectandwant.

Thereareonlyafewinitialhandshakepacketsthataresentintheclearbeforetheencryptionprotocolshavebeennegotiated.

Secure

11

EarlierdataQUICoffersboth0-RTTand1-RTThandshakesthatreducethetimeittakestonegotiateandsetupanewconnection.Comparewiththe3-wayhandshakeofTCP.

Inadditiontothat,QUICoffers"earlydata"supportfromthegetgowhichisdonetoallowmoredataanditisusedmoreeasilythanTCPFastOpen.

Withthestreamconcept,anotherlogicalconnectiontothesamehostcanbedoneatoncewithouthavingtowaitfortheexistingonetoendfirst.

TCPFastOpenisproblematicTCPFastOpenwaspublishedasRFC7413inDecember2014andthatspecificationdescribeshowapplicationscanpassdatatotheservertobedeliveredalreadyinthefirstTCPSYNpacket.

Actualsupportforthisfeatureinthewildhastakentimeandisriddledwithproblemseventodayin2018.TheTCPstackimplementorshavehadissuesandsohaveapplicationstryingtotakeadvantageofthisfeature-bothinknowinginwhichOSversiontotrytoactivateitbutalsoinfiguringouthowtogracefullybackdownanddealwhenproblemsarise.SeveralnetworkshavebeenidentifiedtointerferewithTFOtrafficandtheyhavethusactivelyruinedsuchTCPhandshakes.

Reducedlatency

12

ProcessTheinitialQUICprotocolwasdesignedbyJimRoskindatGoogleandwasinitiallyimplementedin2012,announcedpubliclytotheworldin2013whenGoogle'sexperimentationbroadened.

Backthen,QUICwasstillclaimedtobeanacronymfor"QuickUDPInternetConnections",butthathasbeendroppedsincethen.

Googleimplementedtheprotocolandsubsequentlydeployeditbothintheirwidelyusedbrowser(Chrome)andintheirwidelyusedserver-sideservices(Googlesearch,gmail,youtubeandmore).Theyiteratedprotocolversionsfairlyquicklyandovertimetheyprovedtheconcepttoworkreliablyforavastportionofusers.

InJune2015,thefirstinternetdraftforQUICwassenttotheIETFforstandardization,butittookuntillate2016foraQUICworkinggrouptogetapprovedandstarted.Butthenittookoffimmediatelywithahighdegreeofinterestfrommanyparties.

In2017,numbersquotedbyQUICengineersatGooglementionedthataround7%ofallInternettrafficwerealreadyusingthisprotocol.TheGoogleversionoftheprotocol.

Process

13

IETFTheQUICworkinggroupthatwasestablishedtostandardizetheprotocolwithintheIETFquicklydecidedthattheQUICprotocolshouldbeabletotransferotherprotocolsthan"just"HTTP.Google-QUIConlyevertransportedHTTP-inpracticeittransportedwhatwaseffectivelyHTTP/2frames,usingtheHTTP/2framesyntax.

ItwasalsostatedthatIETF-QUICshouldbaseitsencryptionandsecurityonTLS1.3insteadofthe"custom"approachusedbyGoogle-QUIC.

Inordertosatisfythesend-more-than-HTTPdemand,theIETFQUICprotocolarchitecturewassplitintwoseparatelayers:thetransportQUICandthe"HTTPoverQUIC"layer(thelattersometimesreferredtoas"hq").

Thislayersplit,whileitmaysoundinnocuous,hascausedtheIETF-QUICtodifferquitealotfromtheoriginalGoogle-QUIC.

TheworkinggroupdidhoweversoondecidethatinordertogettheproperfocusandabilitytodeliverQUICversion1ontime,itwouldfocusondeliveringHTTP,leavingnon-HTTPtransportstolaterwork.

InMarch2018whenwestartedworkingonthisbook,theplanwastoshipthefinalspecificationforQUICversion1inNovember2018;thiswaslaterpostponedtoJuly2019.

WhiletheworkonIETF-QUIChasprogressed,theGoogleteamhasincorporateddetailsfromtheIETFversionandhasstartedtoslowlyprogresstheirversionoftheprotocoltowardswhattheIETFversionmightbecome.GooglehascontinuedusingtheirversionofQUICintheirbrowserandservices.

MostnewimplementationsunderdevelopmenthavedecidedtofocusontheIETFversionandarenotcompatiblewiththeGoogleversion.

IETF

14

ExperiencefromHTTP/2TheHTTP/2specificationRFC7540waspublishedinMay2015,justamonthbeforeQUICwasbroughttoIETFforthefirsttime.

WithHTTP/2,thefoundationforchangingHTTPoverthewirewaslaidoutandtheworkinggroupthatcreatedHTTP/2wasalreadyofthemindsetthatthiswouldhelpiteratingtonewHTTPversionsmuchfasterthanithadtakentogotoversion2fromversion1(about16years).

WithHTTP/2,usersandsoftwarestacksgotusedtotheideathatHTTPcannolongerbeassumedtobedonewithatext-basedprotocolinaserialmanner.

HTTP-over-QUICwasrenamedtoHTTP/3inNovember2018.

ExperiencefromHTTP/2

15

StatusTheQUICworkinggrouphasworkedfiercelysincelate2016onspecifyingtheprotocolsandtheplanisnowtohaveitdonebyJuly2019.

AsofNovember2018,therestillhasnotbeenanylargerinteroperabilitytestswithHTTP/3-onlywiththeexistingtwoimplementationsandnoneofthemaredonebyabrowserorapopularopenserversoftware.

TherearefifteenorsodifferentQUICimplementationslistedintheQUICworkinggroups'wikipages,butfarfromallofthemcaninteroperateonthelatestspecdraftrevisions.

ImplementingQUICisnoteasyandtheprotocolhaskeptmovingandchangingevenuptothisdate.

ServersNGINXsupportforQUICandHTTP/3isunderdevelopment.ItisplannedtobereleasedduringNGINX1.17developmentcycle.

TherehavebeennopublicstatementintermsofsupportforQUICfromApache.

ClientsNoneofthelargerbrowservendorshaveyetshippedanyversion,atanystate,thatcanruntheIETFversionofQUICorHTTP/3.

GoogleChromehasshippedwithaworkingimplementationofGoogle'sownQUICversionsincemanyyears,butthatdoesnotinteroperatewiththeIETFQUICprotocolanditsHTTPimplementationisdifferentthanHTTP/3.

MozillaisdevelopingNeqo-aQUICandHTTP/3implementationwritteninRust.NeqoisplannedtobeintegratedinNecko(whichisanetworklibraryusedinmanyMozilla-basedclientapplications-includingFirefox).

curlshippedthefirstexperimentalHTTP/3support(draft-22)inthe7.66.0releaseonSeptember11,2019.curluseseithertheQuichelibraryfromCloudflareorthengtcp2familyoflibrariestogettheworkdone.

ImplementationObstaclesQUICdecidedtouseTLS1.3asthefoundationforthecryptoandsecuritylayertoavoidinventingsomethingnewandinsteadleanonatrustworthyandexistingprotocol.However,whiledoingthis,theworkinggroupalsodecidedthattoreallystreamlinetheuseofTLSinQUIC,itshouldonlyuse"TLSmessages"andnot"TLSrecords"fortheprotocol.

Thismightsoundlikeaninnocuouschange,butthishasactuallycausedasignificanthurdleformanyQUICstackimplementors.ExistingTLSlibrariesthatsupportTLS1.3simplydonothaveAPIsenoughtoexposethisfunctionalityandallowQUICtoaccessit.WhileseveralQUICimplementorscomefromlargerorganizationswhoworkontheirownTLSstackinparallel,thisisnottrueforeveryone.

ThedominantopensourceheavyweightOpenSSLforexample,doesnothaveanyAPIforthis.TheplantoaddressthisseemstohappnintheirPR8797thataimstointroduceanAPIthatisverysimilartotheoneofBoringSSL.

ThiswilleventuallyalsoleadtodeploymentobstaclessinceQUICstackswillneedtoeitherbasethemselvesonotherTLSlibraries,useaseparatepatchedOpenSSLbuildorrequireanupdatetoafutureOpenSSLversion.

KernelsandCPUload

Status

16

BothGoogleandFacebookhavementionedthattheirwidescaledeploymentsofQUICrequireroughlytwicetheamountofCPUthanthesametrafficloaddoeswhenservingHTTP/2overTLS.

Someexplanationsforthisinclude

theUDPpartsinprimarilyLinuxisnotatallasoptimizedastheTCPstackis,sinceithasnottraditionallybeenusedforhighspeedtransferslikethis.

TCPandTLSoffloadingtohardwareexist,butthatismuchrarerforUDPandbasicallynon-existingforQUIC.

TherearereasonstobelievethatperformanceandCPUrequirementswillimproveovertime.

Status

17

ProtocolfeaturesTheQUICprotocolfromahighlevel.

IllustratedbelowistheHTTP/2networkstackontheleftandtheQUICnetworkstackontheright,whenusedasHTTPtransport.

Protocolfeatures

18

TransferprotocoloverUDPQUICisatransferprotocolimplementedontopofUDP.Ifyouwatchyournetworktrafficcasually,youwillseeQUICappearasUDPpackets.

BasedonUDPitalsothenusesUDPportnumberstoidentifyspecificnetworkservicesonagivenIPaddress.

AllknownQUICimplementationsarecurrentlyinuser-space,whichallowsformorerapidevolutionthankernel-spaceimplementationstypicallyallow.

Willitwork?ThereareenterprisesandothernetworksetupsthatblockUDPtrafficonotherportsthan53(usedforDNS).OthersthrottlesuchdatainwaysthatmakesQUICperformworsethanTCPbasedprotocols.Thereisnoendtowhatsomeoperatorsmaydo.

Fortheforeseeablefuture,alluseofQUIC-basedtransportswillprobablyhavetobeabletogracefullyfall-backtoanother(TCP-based)alternative.Googleengineershavepreviouslymentionedmeasuredfailureratesinthelowsingle-digitpercentages.

Willitimprove?ChancesarethatifQUICprovestobeavaluableadditiontotheInternetworld,peoplewillwanttouseitandtheywillwantittofunctionintheirnetworksandthencompaniesmaystarttoreconsidertheirobstacles.DuringtheyearsthedevelopmentofQUIChasprogressed,thesuccessrateforestablishingandusingQUICconnectionsacrosstheInternethasincreased.

UDP

19

ReliabledatatransfersWhileUDPisnotareliabletransport,QUICaddsalayerontopofUDPthatintroducesreliability.Itoffersre-transmissionsofpackets,congestioncontrol,pacingandtheotherfeaturesotherwisepresentinTCP.

DatasentoverQUICfromoneend-pointwillappearintheotherendsoonerorlater,aslongastheconnectionismaintained.

Reliable

20

MultiplestreamswithinconnectionsSimilartoSCTP,SSHandHTTP/2,QUICfeaturesseparatelogicalstreamswithinthephysicalconnections.Anumberofparallelstreamsthatcantransferdatasimultaneouslyoverasingleconnectionwithoutaffectingtheotherstreams.

Aconnectionisanegotiatedsetupbetweentwoend-pointssimilartohowaTCPconnectionworks.AQUICconnectionismadetoaUDPportandIPaddress,butonceestablishedtheconnectionisassociatedbyits"connectionID".

Overanestablishedconnection,eithersidecancreatestreamsandsenddatatotheotherend.Streamsaredeliveredin-orderandtheyarereliable,butdifferentstreamsmaybedeliveredout-of-order.

QUICoffersflowcontrolonbothconnectionandstreams.

Seefurtherdetailsinconnectionsandstreamssections

Streams

21

InorderdeliveryQUICguaranteesin-orderdeliveryofstreams,butnotbetweenstreams.Thismeansthateachstreamwillsenddataandmaintaindataorder,buteachstreammayreachthedestinationinadifferentorderthantheapplicationsentit!

Forexample:streamAandBaretransferredfromaservertoaclient.StreamAisstartedfirstandthenstreamB.InQUIC,alostpacketonlyaffectsthestreamtowhichthelostpacketbelongs.IfstreamAlosesapacketbutstreamBdoesnot,streamBmaycontinueitstransfersandcompletewhilestreamA'slostpacketisre-transmitted.ThiswasnotpossiblewithHTTP/2.

IllustratedherewithoneyellowandonebluestreamsentbetweentwoQUICend-pointsoverasingleconnection.Theyareindependentandmayarriveinadifferentorder,buteachstreamisreliablydeliveredtotheapplicationinorder.

InOrder

22

FasthandshakesQUICoffersboth0-RTTand1-RTTconnectionsetups,meaningthatatbestQUICneedsnoextraround-tripsatallwhensettingupanewconnection.Thefasterofthosetwo,the0-RTThandshake,onlyworksiftherehasbeenapreviousconnectionestablishedtoahostandasecretfromthatconnectionhasbeencached.

EarlydataQUICallowsaclienttoincludedataalreadyinthe0-RTThandshake.Thisfeatureallowsaclienttodeliverdatatothepeerasfastasitpossiblycan,andthatthenofcourseallowstheservertorespondandsenddatabackevensooner.

Fasthandshakes

23

TLS1.3ThetransportsecurityusedinQUICisusingTLS1.3(RFC8446)andthereareneveranyunencryptedQUICconnections.

TLS1.3hasseveraladvantagescomparedtoolderTLSversionsbutaprimaryreasonforusingitinQUICisthat1.3changedthehandshaketorequirefewerroundtrips.Itreducesprotocollatency.

TheGooglelegacyversionofQUICusedacustomcrypto.

TLS1.3

24

TransportandapplicationlevelTheIETFQUICprotocolisatransportprotocol,ontopofwhichotherapplicationprotocolscanbeused.TheinitialapplicationlayerprotocolisHTTP/3(h3).

Thetransportlayersupportsconnectionsandstreams.

ThelegacyGoogleversionofQUIChadtransportandHTTPgluedtogetherintoonesingledo-it-allandwasamorespecial-purposesend-http/2-frames-over-udpprotocol.

Transportandapplication

25

HTTP/3overQUICTheHTTPlayer,calledHTTP/3,doesHTTP-styletransports,includingHTTPheadercompressionusingQPACK-whichissimilartotheHTTP/2compressionnamedHPACK.

TheHPACKalgorithmdependsonanordereddeliveryofstreamssoitwasnotpossibletoreuseitforHTTP/3withoutmodificationssinceQUICoffersstreamsthatcanbedeliveredoutoforder.QPACKcanbeseenastheQUIC-adaptedversionofHPACK.

HTTP/3overQUIC

26

Non-HTTPoverQUICTheworkonsendingprotocolsotherthanHTTPoverQUIChasbeenpostponeduntilafterQUICversion1hasshipped.

Non-HTTPoverQUIC

27

HowQUICworksWithoutexplainingtheexactbitsandbytesonthewire,thissectiondescribeshowthefundamentalbuildingblocksoftheQUICtransportprotocolwork.IfyouwanttoimplementyourownQUICstack,thisdescriptionshouldgiveyouageneralunderstanding,butforallthedetails,refertotheactualIETFInternetDraftsandRFCs.

1. Setupaconnection2. ...thatincludesTLSsecurity3. Thenusestreams

HowQUICworks

28

ConnectionsAQUICconnectionisasingleconversationbetweentwoQUICendpoints.QUIC'sconnectionestablishmentcombinesversionnegotiationwiththecryptographicandtransporthandshakestoreduceconnectionestablishmentlatency.

Toactuallysenddataoversuchaconnection,oneormorestreamshavetobecreatedandused.

ConnectionIDEachconnectionpossessesasetofconnectionidentifiers,orconnectionIDs,eachofwhichcanbeusedtoidentifytheconnection.ConnectionIDsareindependentlyselectedbyendpoints;eachendpointselectstheconnectionIDsthatitspeeruses.

TheprimaryfunctionoftheseconnectionIDsistoensurethatchangesinaddressingatlowerprotocollayers(UDP,IP,andbelow)donotcausepacketsforaQUICconnectiontobedeliveredtothewrongendpoint.

BytakingadvantageoftheconnectionID,connectionscanthusmigratebetweenIPaddressesandnetworkinterfacesinwaysTCPnevercould.Forinstance,migrationallowsanin-progressdownloadtomovefromacellularnetworkconnectiontoafasterwificonnectionwhentheusermovestheirdeviceintoalocationofferingwifi.Similarly,thedownloadcanproceedoverthecellularconnectionifwifibecomesunavailable.

PortnumbersQUICisbuiltatopUDP,soa16bitportnumberfieldisusedtodifferentiateincomingconnections.

VersionnegotiationAnQUICconnectionrequestoriginatingfromaclientwilltelltheserverwhichQUICprotocolversionitwantstospeak,andtheserverwillrespondwithalistofsupportedversionsfortheclienttoselectfrom.

Connections

29

ConnectionsuseTLSImmediatelyaftertheinitialpacketsettingupaconnection,theinitiatorsendsacryptoframethatstartssettingupthesecurelayerhandshake.ThesecuritylayerusesTLS1.3security.

Thereisnowaytoopt-outoravoidusingTLSforaQUICconnection.Theprotocolisdesignedtobehardformiddle-boxestotamperwith,inordertohelppreventossificationoftheprotocol.

ConnectionsuseTLS

30

StreamsStreamsinQUICprovidealightweight,orderedbyte-streamabstraction.

TherearetwobasictypesofstreaminQUIC:

Unidirectionalstreamscarrydatainonedirection:fromtheinitiatorofthestreamtoitspeer.

Bidirectionalstreamsallowfordatatobesentinbothdirections.

Eithertypeofstreamcanbecreatedbyeitherendpoint,canconcurrentlysenddatainterleavedwithotherstreams,andcanbecanceled.

TosenddataoveraQUICconnection,oneormorestreamsareused.

FlowcontrolStreamsareindividuallyflowcontrolled,allowinganendpointtolimitmemorycommitmentandtoapplybackpressure.Thecreationofstreamsisalsoflowcontrolled,witheachpeerdeclaringthemaximumstreamIDitiswillingtoacceptatagiventime.

StreamIdentifiersStreamsareidentifiedbyanunsigned62-bitinteger,referredtoastheStreamID.TheleastsignificanttwobitsoftheStreamIDareusedtoidentifythetypeofstream(unidirectionalorbidirectional)andtheinitiatorofthestream.

Theleastsignificantbit(0x1)oftheStreamIDidentifiestheinitiatorofthestream.Clientsinitiateeven-numberedstreams(thosewiththeleastsignificantbitsetto0);serversinitiateodd-numberedstreams(withthebitsetto1).

Thesecondleastsignificantbit(0x2)oftheStreamIDdifferentiatesbetweenunidirectionalstreamsandbidirectionalstreams.Unidirectionalstreamsalwayshavethisbitsetto1andbidirectionalstreamshavethisbitsetto0.

StreamconcurrencyQUICallowsforanarbitrarynumberofstreamstooperateconcurrently.AnendpointlimitsthenumberofconcurrentlyactiveincomingstreamsbylimitingthemaximumstreamID.

ThemaximumstreamIDisspecifictoeachendpointandappliesonlytothepeerthatreceivesthesetting.

SendingandReceivingDataEndpointsusestreamstosendandreceivedata.Thatisafteralltheirultimatepurpose.Streamsareanorderedbyte-streamabstraction.Separatestreamsarehowevernotnecessarilydeliveredintheoriginalorder.

StreamPrioritizationStreammultiplexinghasasignificanteffectonapplicationperformanceifresourcesallocatedtostreamsarecorrectlyprioritized.Experiencewithothermultiplexedprotocols,suchasHTTP/2,showsthateffectiveprioritizationstrategieshaveasignificantpositiveimpactonperformance.

Streams

31

QUICitselfdoesnotprovideframesforexchangingprioritizationinformation.InsteaditreliesonreceivingpriorityinformationfromtheapplicationthatusesQUIC.ProtocolsthatuseQUICareabletodefineanyprioritizationschemethatsuitstheirapplicationsemantics.

WhenHTTP/3isusedoverQUIC,theprioritizationisdoneintheHTTPlayer.

Streams

32

0-RTTToreducethetimerequiredtoestablishanewconnection,aclientthathaspreviouslyconnectedtoaservermaycachecertainparametersfromthatconnectionandsubsequentlysetupa0-RTTconnectionwiththeserver.Thisallowstheclienttosenddataimmediately,withoutwaitingforahandshaketocomplete.

0-RTT

33

SpinBitOneoftheperhapslongestdesigndiscussionswithintheQUICworkinggroupthathasbeenthesubjectofseveralhundredmailsandhoursofdiscussionsconcernsasinglebit:theSpinBit.

TheproponentsofthisbitinsistthatthereisaneedforoperatorsandpeopleonthepathbetweentwoQUICendpointstobeabletomeasurelatency.

Theopponentstothisfeaturedonotlikethepotentialinformationleak.

SpinningabitBothendpoints,theclientandtheserver,maintainaspinvalue,0or1,foreachQUICconnection,andtheysetthespinbitonpacketsitsendsforthatconnectiontotheappropriatevalue.

Bothsidesthensendoutpacketswiththatspinbitsettothesamevalueforaslongasoneroundtriplastsandthenittogglesthevalue.Theeffectisthenapulseofonesandzeroesinthatbitfieldthatobserverscanmonitor.

Thismeasuringonlyworkswhenthesenderisneitherapplicationnorflowcontrollimitedandpacketreorderingoverthenetworkcanalsomakethedatanoisy.

SpinBit

34

User-spaceImplementingatransportprotocolinuser-spacehelpsenablequickiterationoftheprotocol,asitiscomparativelyeasytoevolvetheprotocolwithoutnecessitatingthatclientsandserversupdatetheiroperatingsystemkerneltodeploynewversions.

NothinginherentinQUICpreventsitfrombeingimplementedandofferedbyoperatingsystemkernelsinthefuture,shouldsomeonefindthatagoodidea.

Manyimplementations

Oneobviouseffectofimplementinganewtransportprotocolinuser-spaceisthatwecanexpecttoseemanyindependentimplementations.

Differentapplicationsarelikelytoinclude(orlayeratop)differentHTTP/3andQUICimplementationsfortheforeseeablefuture.

Userspace

35

APIOneofthesuccessfactorsforregularTCPandprogramsusingthat,isthestandardizedsocketAPI.IthaswelldefinedfunctionalityandusingthisAPIyoucanmoveprogramsbetweenmanydifferentoperatingsystemsasTCPworksthesame.

QUICisnotthere.ThereisnostandardAPIforQUIC.

WithQUIC,youneedtopickoneoftheexistinglibraryimplementationsandstickwithitsAPI.Itmakesapplications"lockedin"toasinglelibrarytosomeextent.ChangingtoanotherlibrarymeansanotherAPIandthatmightinvolvealotofwork.

Also,sinceQUICistypicallyimplementedinuser-space,itcan'teasilyjustextendthesocketAPIorappearsimilartohowexistingTCPandUDPfunctionalitydo.UsingQUICwillmeanusinganotherAPIthanthesocketAPI.

API

36

HTTP/3Asmentionedpreviously,thefirstandprimaryprotocoltotransportoverQUICisHTTP.

MuchlikeHTTP/2wasonceintroducedtotransportHTTPoverthewireinacompletelynewway,HTTP/3isyetagainintroducinganewwaytosendHTTPoverthenetwork.

HTTPstillmaintainsthesameparadigmsandconceptslikebefore.Thereareheadersandabody,thereisarequestandaresponse.Thereareverbs,cookiesandcaching.WhatprimarilychangeswithHTTP/3ishowthebitsgetssentovertotheothersideofthecommunication.

InordertodoHTTPoverQUIC,changeswererequiredandtheresultsofthisiswhatwenowcallHTTP/3.ThesechangeswererequiredbecauseofthedifferentnaturethatQUICprovidesasopposedtoTCP.Thesechangesinclude:

InQUICthestreamsareprovidedbythetransportitself,whileinHTTP/2thestreamsweredonewithintheHTTPlayer.

Duetothestreamsbeingindependentofeachother,theheadercompressionprotocolusedforHTTP/2couldnotbeusedwithoutitcausingaheadofblocksituation.

QUICstreamsareslightlydifferentthanHTTP/2streams.TheHTTP/3sectionwilldetailthissomewhat.

HTTP/3

37

HTTPS://URLsHTTP/3willbeperformedusingHTTPS://URLs.TheworldisfulloftheseURLsandithasbeendeemedimpracticalanddownrightunreasonabletointroduceanotherURLschemeforthenewprotocol.MuchlikeHTTP/2didnotneedanewscheme,neitherwillHTTP/3.

TheaddedcomplexityintheHTTP/3situationishoweverthatwhereHTTP/2wasacompletelynewwayoftransportingHTTPoverthewire,itwasstillbasedonTLSandTCPlikeHTTP/1was.ThefactthatHTTP/3isdoneoverQUICchangesthingsinafewimportantaspects.

Legacy,clear-text,HTTP://URLswillbeleftas-isandasweproceedfurtherintoafuturewithmoresecuretransferstheywillprobablybecomelessandlessfrequentlyused.RequeststosuchURLswillsimplynotbeupgradedtouseHTTP/3.InrealitytheyrarelyupgradetoHTTP/2either,butforotherreasons.

InitialconnectionThefirstconnectiontoafresh,notpreviouslyvisitedhostforaHTTPS://URLprobablyhastobedoneoverTCP(possiblyinadditiontoaparallelattempttoconnectviaQUIC).ThehostmightbealegacyserverwithoutQUICsupportortheremightbeamiddleboxinbetweensettingupobstaclespreventingaQUICconnectionfromsucceeding.

AmodernclientandserverwouldpresumablynegotiateHTTP/2inthefirsthandshake.WhentheconnectionhasbeensetupandtheserverrespondstoaclientHTTPrequest,theservercantelltheclientaboutitssupportofandpreferenceforHTTP/3.

HTTPS://URLs

38

Alt-svcThealternateservice(Alt-svc:)headeranditscorrespondingALT-SVCHTTP/2framearenotspecificallycreatedforQUICorHTTP/3.Theyarepartofanalreadydesignedandcreatedmechanismforaservertotellaclient:"look,IrunthesameserviceonTHISHOSTusingTHISPROTOCOLonTHISPORT".SeedetailsinRFC7838.

AclientthatreceivessuchanAlt-svcresponseisthenadvisedto,ifitsupportsandwantsto,connecttothatgivenotherhostinparallelinthebackground-usingthespecifiedprotocol-andifitissuccessfulswitchitsoperationsovertothatinsteadoftheinitialconnection.

IftheinitialconnectionusesHTTP/2orevenHTTP/1,theservercanrespondandtelltheclientthatitcanconnectbackandtryHTTP/3.Itcouldbetothesamehostortoanotheronethatknowshowtoservethatorigin.TheinformationgiveninsuchanAlt-svcresponsehasanexpirytimer,makingclientsabletodirectsubsequentconnectionsandrequestsdirectlytothealternativehostusingthesuggestedalternativeprotocol,foracertainperiodoftime.

ExampleAnHTTPserverincludesanAlt-Svc:headerinitsresponse:

Alt-Svc:h3=":50781"

ThisindicatesthatHTTP/3isavailableonUDPport50781atthesamehostnamethatwasusedtogetthisresponse.

AclientcanthenattempttosetupaQUICconnectiontothatdestinationandifsuccessful,continuecommunicatingwiththeoriginlikethatinsteadoftheinitialHTTPversion.

BootstrapwithAlt-svc

39

QUICstreamsandHTTP/3HTTP/3ismadeforQUICsoittakesfulladvantageofQUIC'sstreams,whereHTTP/2hadtodesignitsentirestreamandmultiplexingconceptofitsownontopofTCP.

HTTPrequestsdoneoverHTTP/3useaspecificsetofstreams.

HTTP/3framesHTTP/3meanssettingupQUICstreamsandsendingoverasetofframestotheotherend.There'sbutasmallfixednumber(actuallynineonDecember18th,2018!)ofknownframesinHTTP/3.Themostimportantonesareprobably:

HEADERS,thatsendscompressedHTTPheadersDATA,sendsbinarydatacontentsGOAWAY,pleaseshutdownthisconnection

HTTPRequestTheclientsendsitsHTTPrequestonaclient-initiatedbidirectionalQUICstream.

ArequestconsistsofasingleHEADERSframeandmightoptionallybefollowedbyoneortwootherframes:aseriesofDATAframesandpossiblyafinalHEADERSframefortrailers.

Aftersendingarequest,aclientclosesthestreamforsending.

HTTPResponseTheserversendsbackitsHTTPresponseonthebidirectionalstream.AHEADERSframe,aseriesofDATAframesandpossiblyatrailingHEADERSframe.

QPACKheadersTheHEADERSframescontainHTTPheaderscompressedusingtheQPACKalgorithm.QPACKissimilarinstyletotheHTTP/2compressioncalledHPACK(RFC7541),butmodifiedtoworkwithstreamsdeliveredoutoforder.

QPACKitselfusestwoadditionalunidirectionalQUICstreamsbetweenthetwoend-points.Theyareusedtocarrydynamictableinformationineitherdirection.

QUICstreamsandHTTP/3

40

HTTP/3PrioritizationOneoftheHTTP/3streamframesiscalledPRIORITY.ItisusedtosetpriorityanddependencyonastreaminawaysimilartohowitworksinHTTP/2.

Theframecansetaspecificstreamtodependonanotherspecificstreamanditcanseta"weight"onagivenstream.

Adependentstreamshouldonlybeallocatedresourcesifeitherallofthestreamsthatitdependsonareclosedoritisnotpossibletomakeprogressonthem.

Astreamweightisvaluebetween1and256anditisspecifiedthatstreamswiththesameparentshouldbeallocatedresourcesproportionallybasedontheirweight.

Prioritization

41

HTTP/3ServerpushHTTP/3serverpushissimilartowhatisdescribedinHTTP/2(RFC7540),butusesdifferentmechanisms.

Aserverpushiseffectivelytheresponsetoarequestthattheclientneversent!

Serverpushesareonlyallowedtohappeniftheclientsidehasagreedtothem.InHTTP/3theclientevensetsalimitforhowmanypushesitacceptsbyinformingtheserverwhatthemaxpushstreamIDis.Goingoverthatlimitwillcauseaconnectionerror.

Iftheserverdeemsitlikelythattheclientwantsanextraresourcethatithasn'taskedforbutoughttohaveanyway,itcansendaPUSH_PROMISEframe(overtherequeststream)showingwhattherequestlookslikethatthepushisaresponseto,andthensendthatactualresponseoveranewstream.

Evenwhenpusheshavebeensaidtobeacceptablebytheclientbefore-hand,eachindividualpushedstreamcanstillbecanceledatanytimeiftheclientdeemsthatsuitable.ItthensendsaCANCEL_PUSHframetotheserver.

ProblematicEversincethisfeaturewasfirstdiscussedintheHTTP/2developmentandlateraftertheprotocolshippedandhasbeendeployedovertheInternet,thisfeaturehasbeendiscussed,dislikedandpoundedupincountlessdifferentwaysinordertogetittobecomeuseful.

Pushingisnever"free",sincewhileitsavesahalfround-tripitstillusesbandwidth.Itisoftenhardorimpossiblefortheserver-sidetoactuallyknowwithahighlevelofcertaintyifaresourceshouldbepushedornot.

Serverpush

42

HTTP/3comparedtoHTTP/2HTTP/3isdesignedforQUIC,whichisatransportprotocolthathandlesstreamsbyitself.

HTTP/2isdesignedforTCP,andthereforehandlesstreamsintheHTTPlayer.

SimilaritiesThetwoprotocolsofferclientsvirtuallyidenticalfeaturesets.

Bothprotocolsofferstreams

Bothprotocolsofferserverpushsupport

Bothprotocolshaveheadercompression,andQPACKandHPACKaresimilarindesign.

Bothprotocolsoffermultiplexingoverasingleconnectionusingstreams

Bothprotocolsdoprioritizationonstreams

DifferencesThedifferencesareinthedetailsandprimarilytherethankstoHTTP/3'suseofQUIC:

HTTP/3hasbetterandmorelikelytoworkearlydatasupportthankstoQUIC's0-RTThandshakes,whileTCPFastOpenandTLSusuallysendslessdataandoftenfacesproblems.

HTTP/3hasmuchfasterhandshakesthankstoQUICvsTCP+TLS.

HTTP/3doesnotexistinaninsecureorunencryptedversion.HTTP/2canbeimplementedandusedwithoutHTTPS-evenifthisisrareontheInternet.

HTTP/2canbenegotiateddirectlyinaTLShandshakewiththeALPNextension,whileHTTP/3isoverQUICsoitneedsanAlt-Svc:headerresponsefirsttoinformtheclientaboutthisfact.

ComparisonwithHTTP/2

43

Criticism

UDPwillneverworkAlotofenterprises,operatorsandorganizationsblockorrate-limitUDPtrafficoutsideofport53(usedforDNS)sinceithasinrecentdaysmostlybeenabusedforattacks.Inparticular,someoftheexistingUDPprotocolsandpopularserverimplementationsforthemhavebeenvulnerableforamplificationattackswhereoneattackercanmakeahugeamountofoutgoingtraffictotargetinnocentvictims.

QUIChasbuilt-inmitigationagainstamplificationattacksbyrequiringthattheinitialpacketmustbeatleast1200bytesandbyrestrictionintheprotocolthatsaysthataservermustnotsendmorethanthreetimesthesizeoftherequestinresponsewithoutreceivingapacketfromtheclientinresponse.

UDPisslowinkernelsThisseemstobethetruth,atleasttodayin2018.WecanofcoursenottellhowthiswilldevelopandhowmuchofthisissimplytheresultofUDPtransferperformancenothavingbeenindevelopers'focusformanyyears.

Formostclients,this"slowness"isprobablyneverevennoticeable.

QUICtakestoomuchCPUSimilartothe"UDPisslow"remarkabove,thisispartlybecausetheTCPandTLSusageoftheworldhashadalongertimetomature,improveandgethardwareassistance.

Therearereasonstoexpectthistoimproveovertime.ThequestionishowmuchthisextraCPUusagewillhurtdeployers.

ThisisjustGoogleNoitisnot.GooglebroughttheinitialspectotheIETFafterhavingproved,onalargeInternet-widescale,thatdeployingthisstyleofprotocoloverUDPactuallyworksandperformswell.

Sincethen,individualsfromalargenumberofcompaniesandorganizationshaveworkedinthevendor-neutralorganizationIETFtoputtogetherastandardtransportprotocoloutofit.Inthatwork,Googleemployeeshaveofcoursebeenparticipating,butsohaveemployeesfromalargenumberofothercompaniesthatareinterestedinfurtheringthestateoftransportprotocolsontheInternet,includingMozilla,Fastly,Cloudflare,Akamai,Microsoft,FacebookandApple.

ThisistoosmallofanimprovementThatisnotreallyacritiquebutanopinion.Maybeitis,andmaybeitistoolittleofanimprovementsocloseintimesinceHTTP/2wasshipped.

HTTP/3islikelytoperformmuchbetterinpacketloss-riddennetworks,itoffersfasterhandshakessoitwillimprovelatencybothasperceivedandactual.ButisthatenoughofbenefitstomotivatepeopletodeployHTTP/3supportontheirserversandfortheirservices?Timeandfutureperformancemeasurementswillsurelytell!

Commoncriticism

44

ThespecificationsHereisacollectionofthelatestofficialdraftsforthevariouspartsandcomponentsofQUICandHTTP/3.

InvariantsVersion-IndependentPropertiesofQUIC

TransportQUIC:AUDP-BasedMultiplexedandSecureTransport

RecoveryQUICLossDetectionandCongestionControl

TLSUsingTLStoSecureQUIC

HTTPHypertextTransferProtocolVersion3(HTTP/3)

QPACKQPACK:HeaderCompressionforHTTP/3

Thespecifications

45

QUICv2InordertogetthemostpossiblyfocusonthecoreQUICprotocolandbeabletoshipitontime,severalfeaturesthatwereoriginallyplannedtobepartofthecoreprotocolhavebeenpostponedandarenowplannedtoinsteadgetdoneinasubsequentQUICversion.QUICversion2orbeyond.

Theauthorofthisdocumentdoeshoweverhavearatherfaultycrystalballsowecannottellforsureexactlywhatfeatureswillorwillnotappearinversion2.Wecanhowevermentionsomeofthefeaturesandthingsthatexplicitlyhavebeenremovedfromthev1worktobe"workedonlater"andthatthenpossiblymightappearinaversion2.

ForwardErrorCorrectionForwarderrorcorrection(FEC)isamethodofobtainingerrorcontrolindatatransmissioninwhichthetransmittersendsredundantdataandthereceiverrecognizesonlytheportionofthedatathatcontainsnoapparenterrors.

GoogleexperimentedwiththisintheiroriginalQUICworkbutitwassubsequentlyremovedagainsincetheexperimentsdidnotturnoutwell.ThisfeatureissubjectfordiscussionforQUICv2butprobablytakesforsomeonetoprovethatitactuallycanbeausefuladditionwithouttoomuchpenalty.

MultipathMultipathmeansthatthetransportcanbyitselfusemultiplenetworkpathstomaximizeresourceusageandincreaseredundancy.

TheSCTPproponentsoftheworldliketomentionthatSCTPalreadyfeaturesmultipathandsodoesmodernTCP.

UnreliabledataIthasbeendiscussedtooffer"unreliable"streamsasanoption,thatwouldthenallowQUICtoalsoreplaceUDP-styleapplications.

Non-HTTPadaptionsDNSoverQUICwasoneoftheearlymentionednon-HTTPprotocolsthatjustmightgetsomeattentiononceQUICv1andHTTP/3ship.ButwithanewtransportlikethishavingbeenbroughtontotheworldIcannotimaginethatitwillstopthere.

QUICv2

46