agentzh's Nginx Tutorials (ver 2016.07.21) - OpenResty · agentzh's Nginx Tutorials (version...
Transcript of agentzh's Nginx Tutorials (ver 2016.07.21) - OpenResty · agentzh's Nginx Tutorials (version...
agentzh'sNginxTutorials(version2016.07.21)TableofContents
ForewordWritingPlanfortheTutorialsNginxVariables(01)NginxVariables(02)NginxVariables(03)NginxVariables(04)NginxVariables(05)NginxVariables(06)NginxVariables(07)NginxVariables(08)NginxDirectiveExecutionOrder(01)NginxDirectiveExecutionOrder(02)NginxDirectiveExecutionOrder(03)NginxDirectiveExecutionOrder(04)NginxDirectiveExecutionOrder(05)NginxDirectiveExecutionOrder(06)NginxDirectiveExecutionOrder(07)NginxDirectiveExecutionOrder(08)NginxDirectiveExecutionOrder(09)NginxDirectiveExecutionOrder(10)
ForewordI'vebeendoingalotofworkintheNginxworldoverthelastfewyearsandI'vealsobeenthinkingaboutwritingaseriesoftutorial-likearticlestoexplaintomorepeoplewhatI'vedoneandwhatI'velearnedinthisarea.NowIhavefinallydecidedtopostserialarticlestotheSinaBloghttp://blog.sina.com.cn/openrestyinChinese.Everyarticlewillroughlycoverasingletopicandwillbeinarathercasualstyle.ButatsomepointinthefutureImayrestructurethearticlesandtheirstyleinordertoturnthemintoa"real"book.
Thearticlesaredividedintoseries.Forexample,thefirstseriesis"NginxVariables".EachseriescanbethoughtofasmappingtoachapterintheNginxbookthatImaypublishinthefuture.
ThearticlesareintendedforNginxusersofallexperiencelevels,includinguserswithextensiveApacheandLighttpdexperiencewhomayhaveneverusedNginxbefore.
TheexamplesinthearticlesareatleastcompatiblewithNginx0.8.54.DonottrytheexampleswitholderversionsofNginx.ThelateststableversionofNginxasofthiswritingis1.7.9.
AlloftheNginxmodulesreferencedinthearticlesareproduction-ready.IwillnotbecoveringanyNginxcoremodulesthatareeitherexperimentalorbuggy.Additionally,Iwillbemakingextensiveuseof3rd-partyNginxmodulesintheexamples.Ifit'sinconvenientforyoutodownloadandinstalltheindividualmodulesoneatatimethenIhighlyrecommendthatyoudownloadandinstallthengx_openrestysoftwarebundlethatImaintain.
http://openresty.org/
Allofthemodulesreferencedinthearticles,includingthecoreNginxmodulesthatarenew(butstable),areincludedintheOpenRestybundle.
AprinciplethatIwillbetryingtoadheretoistousesmallconciseexamplestoexplainandvalidatetheconceptsandbehaviorsbeingdescribed.Myhopeisthatitwillhelpthereadertodevelopthegoodhabitofnotacceptingothers'viewpointsorstatementsatfacevaluewithouttestingthemfirst.ThisapproachmayhavesomethingtodowithmyQAbackground.Infact,Ikeeptweakingandcorrectingthearticlesbasedontheresultsofrunningtheexampleswhilewriting.
Theexamplesinthearticlesfallintooneoftwocategories,goodandproblematic.ThepurposeoftheproblematicexamplesistohighlightpotentialpitfallsandotherareaswhereNginxoritsmodulesbehaveinwaysthatreadersmaynotexpect.Problematicexamplesareeasytoidentifybecauseeachlineoftextintheexamplewillbeprefixedwithaquestionmark,i.e.,"?".Hereisanexample:
?server{
?listen8080;
?
?location/bad{
?echo$foo;
?}
?}
Donotreproducethesearticleswithoutexplicitpermissionsfromus.Copyrightreserved.
Iencouragereaderstosendfeedback([email protected]),especiallyconstructivecriticism.
ThesourceforallthearticlesisonGitHub:
http://github.com/agentzh/nginx-tutorials/
Thesourcefilesareundertheen/directory.IamusingalittlemarkuplanguagethatisamixtureofWikiandPODtowritethesearticles.Theyarethe.tutfiles.Youarewelcometocreateforksand/orprovidepatches.
Thee-booksfilesthataresuitableforcellphones,Kindle,iPad/iPhone,SonyReaders,andotherdevicescanbedownloadedfromhere:
http://openresty.org/#eBooks
SpecialthanksgotoKaiWu(kai10k)whokindlytranslatesthesearticlestoEnglish.
agentzhathomeintheFuzhoucity
October30,2011
WritingPlanfortheTutorialsHereliststhetutorialseriesthathavealreadybeenpublishedortobepublished.
GettingStartedwithNginxHowNginxMatchesURIsNginxVariablesNginxDirectiveExecutionOrderNginx'sifisEvilNginxSubrequestsNginxStaticFileServicesNginxLogServicesApplicationGatewaysbasedonNginxReverse-ProxiesbasedonNginxNginxandMemcachedNginxandRedisNginxandMySQLNginxandPostgreSQLApplicationcachingBasedonNginxSecurityandAccessControlinNginxWebServicesBasedonNginxAJAXApplicationsDrivenbyNginxPerformanceTestingforNginxanditsApplicationsStrengthoftheNginxCommunity
TheseriesnamescanroughlycorrespondtothechapternamesinmyfinalNginxbook,buttheyareunlikelytostayexactlythesame.Theactualseriesnamesmaychangeandtherelativeorderoftheseriesmaychangeaswell.
Thelistabovewillbeconstantlyupdatedtoalwaysreflectthelatestplan.
Variablesarevaluecontainers
NginxVariables(01)VariablesasValueContainers
Nginx'sconfigurationfilesuseamicroprogramminglanguage.Manyreal-worldNginxconfigurationfilesareessentiallysmallprograms.Thislanguage'sdesignisheavilyinfluencedbyPerlandBourneShellasfarasIcansee,despitethefactthatitmightnotbeTuring-Completeanditisdeclarativeinmanyplaces.ThisisadistinguishingfeatureofNginx,ascomparedtootherwebserverslikeApacheorLighttpd.Beingaprogramminglanguage,"variables"arethusanaturalpartofit(exceptionsdoexist,ofcourse,asinpurefunctionallanguageslikeHaskell).
VariablesarejustcontainersholdingvariousvaluesinimperativelanguageslikePerl,BourneShell,andC/C++.And"values"canbenumberslike3.14,stringslikehelloworld,orevencomplicatedthingslikereferencestoarraysorhashtablesinthoselanguages.FortheNginxconfigurationlanguage,however,variablescanholdonlyonetypeofvalues,thatis,strings(thereisaninterestingexception:the3rd-partymodulengx_array_varextendsNginxvariablestoholdarrays,butitisimplementedbyencodingaCpointerasabinarystringvaluebehindthescene).
VariableSyntaxandInterpolation
Let'ssayournginx.confconfigurationfilehasthefollowingline:
set$a"helloworld";
Weassignavaluetothevariable$aviathesetconfigurationdirectivecomingfromthestandardngx_rewritemodule.Inparticular,weassignthestringvaluehelloworldto$a.
WecanseethattheNginxvariablenametakesadollarsign($)infrontofit.Thisisrequiredbythelanguagesyntax:wheneverwewanttoreferenceanNginxvariableintheconfigurationfile,wemustadda$prefix.ThislooksveryfamiliartothosePerlandPHPprogrammers.
SuchvariableprefixmodifiersmaydiscomfortsomeJavaandC#programmers,thisnotationdoeshaveanobviousadvantagethough,thatis,variablescanbeembeddeddirectlyintoastringliteral:
set$ahello;
set$b"$a,$a";
HereweusethevalueoftheexistingNginxvariable$atoconstructthevalueforthevariable$b.Soafterthesetwodirectivescompleteexecution,thevalueof$aishello,and$bishello,hello.Thistechniqueiscalled"variableinterpolation"inthePerlworld,whichmakesad-hocstringconcatenationoperatorsnolongerthatnecessary.Let'susethesametermfortheNginxworldfromnowon.
Let'sseeanothercompleteexample:
server{
listen8080;
location/test{
set$foohello;
echo"foo:$foo";
}
}
Thisexampleomitsthehttpdirectiveandeventsconfigurationblocksintheouter-mostscopeforbrevity.Torequestthis/testinterfaceviacurl,anHTTPclientutility,onthecommandline,weget
$curl'http://localhost:8080/test'
foo:hello
Hereweusetheechodirectiveofthe3rdpartymodulengx_echotoprintoutthevalueofthe$foovariableastheHTTPresponse.
Apparentlytheargumentsoftheechodirectivedoessupport"variableinterpolation",butwecannottakeitforgrantedforotherdirectives.Becausenotalltheconfigurationdirectivessupport"variableinterpolation"anditisinfactuptotheimplementationofthedirectiveinthatmodule.Alwayslookupthedocumentationtobesure.
Escaping"$"
We'vealreadylearnedthatthe$characterisspecialanditservesasthevariablenameprefix,butnowconsiderthatwewanttooutputaliteral$characterviatheechodirective.Thefollowingnaiveexampledoesnotworkatall:
?:nginx
?location/t{
?echo"$";
?}
Wewillgetthefollowingerrormessagewhileloadingthisconfiguration:
[emerg]invalidvariablenamein...
ObviouslyNginxtriestoparse$"asavariablename.Isthereawaytoescape$inthestringliteral?Theansweris"no"(itisstillthecaseinthelatestNginxstablerelease1.2.7)andIhavebeenhopingthatwecouldwritesomethinglike$$toobtainaliteral$.
Luckily,workaroundsdoexistandhereisoneproposedbyMaximDounin:firstweassigntoavariablealiteralstringcontainingadollarsigncharacterviaaconfigurationdirectivethatdoesnotsupport"variableinterpolation"(rememberthatnotallthedirectivessupport"variableinterpolation"?),andthenreferencethisvariablelaterwheneverweneedadollarsign.Hereissuchanexampletodemonstratetheidea:
geo$dollar{
default"$";
}
server{
listen8080;
location/test{
echo"Thisisadollarsign:$dollar";
}
}
Let'stestitout:
$curl'http://localhost:8080/test'
Thisisadollarsign:$
Herewemakeuseofthegeodirectiveofthestandardmodulengx_geotoinitializethe$dollarvariablewiththestring"$",thereaftervariable$dollarcanbeusedinplacesthatrequireadollarsign.Thisworksbecausethegeodirectivedoesnotsupport"variableinterpolation"atall.However,thengx_geomoduleisoriginallydesignedtosetaNginxvariabletodifferentvaluesaccordingtotheremoteclientaddress,andinthisexample,wejustabuseittoinitializethe$dollarvariablewiththestring"$"unconditionally.
DisambiguatingVariableNames
Thereisaspecialcasefor"variableinterpolation",thatis,whenthevariablenameisfolloweddirectlybycharactersallowedinvariablenames(likeletters,digits,andunderscores).Insuchcases,wecanuseaspecialnotationtodisambiguatethevariablenamefromthesubsequentliteralcharacters,forinstance,
server{
listen8080;
location/test{
set$first"hello";
echo"${first}world";
}
}
Herethevariable$firstisconcatenatedwiththeliteralstringworld.Ifitwerewrittendirectlyas"$firstworld",Nginx's"variableinterpolation"engine(alsoknownasthe"scriptengine")wouldtrytoaccessthevariable$firstworldinsteadof$first.Toresolvetheambiguityhere,curlybracesmustbeusedaroundthevariablename(excludingthe$prefix),asin${first}.Let'stestthissample:
$curl'http://localhost:8080/test
helloworld
VariableDeclarationandCreation
InlanguageslikeC/C++,variablesmustbedeclared(orcreated)beforetheycanbeusedsothatthecompilercanallocatestorageandperformtypecheckingatcompile-time.Similarly,NginxcreatesalltheNginxvariableswhileloadingtheconfigurationfile(orinotherwords,at"configurationtime"),thereforeNginxvariablesarealsorequiredtobedeclaredsomehow.
FortunatelythesetdirectiveandthegeodirectivementionedabovedohavethesideeffectofdeclaringorcreatingNginxvariablesthattheywillassignvaluestolaterat"requesttime".Ifwedonotdeclareavariablethiswayanduseitdirectlyin,say,theechodirective,wewillgetanerror.Forexample,
?server{
?listen8080;
?
?location/bad{
?echo$foo;
?}
?}
Herewedonotdeclarethe$foovariableandaccessitsvaluedirectlyinecho.Nginxwilljustrefuseloadingthisconfiguration:
[emerg]unknown"foo"variable
Yes,wecannotevenstarttheserver!
Nginxvariablecreationandassignmenthappenatcompletelydifferentphasesalongthetime-line.VariablecreationonlyoccurswhenNginxloadsitsconfiguration.Ontheotherhand,variableassignmentoccurswhenrequestsareactuallybeingserved.ThisalsomeansthatwecannevercreatenewNginxvariablesat"requesttime".
VariableScope
OnceanNginxvariableiscreated,itisvisibletotheentireconfiguration,evenacrossdifferentvirtualserverconfigurationblocks,regardlessoftheplacesitisdeclaredat.Hereisanexample:
server{
listen8080;
location/foo{
echo"foo=[$foo]";
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
}
Herethevariable$fooiscreatedbythesetdirectivewithinlocation/bar,andthisvariableisvisibletotheentireconfiguration,thereforewecanreferenceitinlocation/foowithoutworries.Belowistheresultoftestingthesetwointerfacesviathecurltool.
$curl'http://localhost:8080/foo'
foo=[]
$curl'http://localhost:8080/bar'
foo=[32]
$curl'http://localhost:8080/foo'
foo=[]
Wecanseethattheassignmentoperationisonlyperformedinrequeststhataccesslocation/bar,sincethecorrespondingsetdirectiveisonlyusedinthatlocation.Whenrequestingthe/foointerface,wealwaysgetanemptyvalueforthe$foovariablebecausethatiswhatwegetwhenaccessinganuninitializedvariable.
AnotherimportantcharacteristicthatwecanobservefromthisexampleisthateventhoughthescopeofNginxvariablesistheentireconfiguration,eachrequestdoeshaveitsownversionofallthosevariables'containers.Requestsdonotinterferewitheachothereveniftheyarereferencingavariablewiththesamename.ThisisverymuchlikelocalvariablesinC/C++functionbodies.EachinvocationoftheC/C++functiondoesuseitsownversionofthoselocalvariables(onthestack).
Forinstance,inthissample,werequest/barandthevariable$foogetsthevalue32,whichdoesnotaffectthevalueof$fooinsubsequentrequeststo/foo(itisstilluninitialized!),becausetheycorrespondtodifferentvaluecontainers.
OnecommonmistakeforNginxnewcomersistoregardNginxvariablesassomethingsharedamongalltherequests.EventhoughthescopeofNginxvariablenamesgoacrossconfigurationblocksat"configurationtime",itsvaluecontainer'sscopenevergoesbeyondrequestboundariesat"requesttime".Essentiallyherewedohavetwodifferentkindsofscopehere.
NginxVariables(02)VariableLifetime&InternalRedirection
WealreadyknowthatNginxvariablesareboundtoeachrequesthandledbyNginx,forthisreasontheyhaveexactlythesamelifetimeasthecorrespondingrequest.
Thereisanothercommonmisunderstandingherethough:somenewcomerstendtoassumethatthelifetimeofNginxvariablesisboundtothelocationconfigurationblock.Let'sconsiderthefollowingcounterexample:
server{
listen8080;
location/foo{
set$ahello;
echo_exec/bar;
}
location/bar{
echo"a=[$a]";
}
}
Hereinlocation/fooweusetheecho_execdirective(providedbythe3rd-partymodulengx_echo)toinitiatean"internalredirection"tolocation/bar.The"internalredirection"isanoperationthatmakesNginxjumpfromonelocationtoanotherwhileprocessingarequest.This"jumping"happenscompletelywithintheserveritself.Thisisdifferentfromthose"externalredirections"basedontheHTTP301and302responsesbecausethelatteriscollaboratedexternally,bytheHTTPclients.Also,incaseof"externalredirections",theendusercouldusuallyobservethechangeoftheURLinherwebbrowser'saddressbarwhilethisisnotthecaseforinternalones."Internalredirections"areverysimilartotheexeccommandinBourneShell;itisa"onewaytrip"andneverreturns.AnothersimilarexampleisthegotostatementintheClanguage.
Beingan"internalredirection",therequestaftertheredirectionremainstheoriginalone.Itisjustthecurrentlocationthatischanged,sowearestillusingtheoriginalcopyoftheNginxvariablecontainers.Backtoourexample,thewholeprocesslookslikethis:Nginxfirstassignstothe$avariablethestringvaluehelloviathesetdirectiveinlocation/foo,andthenitissuesaninternalredirectionviatheecho_execdirective,thusleavinglocation/fooandenteringlocation/bar,andfinallyitoutputsthevalueof$a.Becausethevaluecontainerof$aremainsuntouched,wecanexpecttheresponseoutputtobehello.Thetestresultconfirmsthis:
$curllocalhost:8080/foo
a=[hello]
Butwhenaccessing/bardirectlyfromtheclientside,wewillgetanemptyvalueforthe$avariable,sincethisvariablereliesonlocation/footogetinitialized.
Itcanbeobservedthatduringarequest'slifetime,thecopyofNginxvariablecontainersdoesnotchangeatallevenwhenNginxgoesacrossdifferentlocationconfigurationblocks.Herewealsoencountertheconceptof"internalredirections"forthefirsttimeandit'sworthmentioningthattherewritedirectiveofthengx_rewritemodulecanalsobeusedtoinitiate"internalredirections".Forinstance,wecanrewritetheexampleabovewiththerewritedirectiveasfollows:
server{
listen8080;
location/foo{
set$ahello;
rewrite^/bar;
}
location/bar{
echo"a=[$a]";
}
}
It'sfunctionallyequivalenttoecho_exec.Wewilldiscusstherewritedirectiveinmoredepthinlaterchapters,likeinitiating"externalredirections"like301and302.
Toconclude,thelifetimeofNginxvariablecontainersisindeedboundtotherequestbeingprocessed,andisirrelevanttolocation.
NginxBuilt-inVariables
TheNginxvariableswehaveseensofarareall(implicitly)createdbydirectiveslikeset.Weusuallycallsuchvariables"user-definedvaraibles",orsimply"uservariables".ThereisalsoanotherkindofNginxvariablesthatarepre-definedbyeithertheNginxcoreorNginxmodules.Let'scallthiskindofvariables"built-invariables".
$uri&$request_uri
OnecommonuseofNginxbuilt-invariablesistoretrievevarioustypesofinformationaboutthecurrentrequestorresponse.Forinstance,thebuilt-invariable$uriprovidedbyngx_http_coreisusedtofetchthe(decoded)URIofthecurrentrequest,excludinganyquerystringarguments.Anotherexampleisthe$request_urivariableprovidedbythesamemodule,whichisusedtofetchtheraw,non-decodedformoftheURI,includinganyquerystring.Let'slookatthefollowingexample.
location/test{
echo"uri=$uri";
echo"request_uri=$request_uri";
}
Weomittheserverconfigurationblockhereforbrevity.Justasallthosesamplesabove,westilllistentothe8080localport.Inthisexample,weoutputboththe$uriand$request_uriintotheresponsebody.Belowistheresultoftestingthis/testinterfacewithdifferentrequests:
$curl'http://localhost:8080/test'
uri=/test
request_uri=/test
$curl'http://localhost:8080/test?a=3&b=4'
uri=/test
request_uri=/test?a=3&b=4
$curl'http://localhost:8080/test/hello%20world?a=3&b=4'
uri=/test/helloworld
request_uri=/test/hello%20world?a=3&b=4
VariableswithInfiniteNames
Thereisanotherverycommonbuilt-invariablethatdoesnothaveafixedvariablename.Instead,Ithasinfinitevariations.Thatis,allthosevariableswhosenameshavetheprefixarg_,like$arg_fooand$arg_bar.Let'sjustcallitthe$arg_XXX"variablegroup".Forexample,the$arg_namevariableisevaluatedtothevalueofthenameURIargumentforthecurrentrequest.Also,theURIargument'svalueobtainedhereisnotdecodedyet,potentiallycontainingthe%XXsequences.Let'scheckoutacompleteexample:
location/test{
echo"name:$arg_name";
echo"class:$arg_class";
}
ThenwetestthisinterfacewithvariousdifferentURIargumentcombinations:
$curl'http://localhost:8080/test'
name:
class:
$curl'http://localhost:8080/test?name=Tom&class=3'
name:Tom
class:3
$curl'http://localhost:8080/test?name=hello%20world&class=9'
name:hello%20world
class:9
Infact,$arg_namedoesnotonlymatchthenameargumentname,butalsoNAMEorevenName.Thatis,thelettercasedoesnotmatterhere:
$curl'http://localhost:8080/test?NAME=Marry'
name:Marry
class:
$curl'http://localhost:8080/test?Name=Jimmy'
name:Jimmy
class:
Behindthescene,NginxjustconvertstheURIargumentnamesintothepurelower-caseformbeforematchingagainstthenamespecifiedby$arg_XXX.
Ifyouwanttodecodethespecialsequenceslike%20intheURIargumentvalues,thenyoucouldusetheset_unescape_uridirectiveprovidedbythe3rd-partymodulengx_set_misc.
location/test{
set_unescape_uri$name$arg_name;
set_unescape_uri$class$arg_class;
echo"name:$name";
echo"class:$class";
}
Let'scheckouttheactualeffect:
$curl'http://localhost:8080/test?name=hello%20world&class=9'
name:helloworld
class:9
Thespacehasindeedbeendecoded!
Anotherthingthatwecanobservefromthisexampleisthattheset_unescape_uridirectivecanalsoimplicitlycreateNginxuser-definedvariables,justlikethesetdirective.Wewilldiscussthengx_set_miscmoduleinmoredetailinfuturechapters.
Thistypeofvariableslike$arg_XXXpossessesinfinitenumberofpossiblenames,sotheydonotcorrespondtoanyvaluecontainers.Furthermore,suchvariablesarehandledinaveryspecificwaywithintheNginxcore.Itisthusnotpossiblefor3rd-partymodulestointroducesuchmagicalbuilt-invariablesoftheirown.
TheNginxcoreoffersalotofsuchbuilt-invariablesinadditionto$arg_XXX,likethe$cookie_XXXvariablegroupforfetchingHTTPcookievalues,the$http_XXXvariablegroupforfetchingrequestheaders,aswellasthe$sent_http_XXXvariablegroupforretrievingresponseheaders.Wewillnotgointothedetailsforeachofthemhere.Interestedreaderscanrefertotheofficialdocumentationforthengx_http_coremodule.
Read-onlyBuilt-inVariables
Alltheuser-definedvariablesarewritable.Actuallythewaythatwedeclareorcreatesuchvariablessofaristouseaconfiguredirective,likeset,thatperformsvalueassignmentatrequesttime.Butitisnotnecessarilythecaseforbuilt-invariables.
Mostofthebuilt-invariablesareeffectivelyread-only,likethe$uriand$request_urivariablesthatwejustintroducedearlier.Assignmentstosuchread-onlyvariablesmustalwaysbeavoided.Otherwiseitwillleadtounexpectedconsequences,forexample,
?location/bad{
?set$uri/blah;
?echo$uri;
?}
ThisproblematicconfigurationjusttriggersaconfusingerrormessagewhenNginxisstarted:
[emerg]theduplicate"uri"variablein...
Attemptsofwritingtosomeotherread-onlybuilt-invariableslike$arg_XXXwilljustleadtoservercrashesinsomeparticularNginxversions.
NginxVariables(03)WritableBuilt-inVariable$args
Somebuilt-invariablesarewritableaswell.Forinstance,whenreadingthebuilt-invariable$args,wegettheURLquerystringofthecurrentrequest,butwhenwritingtoit,weareeffectivelymodifyingthequerystring.Hereissuchanexample:
location/test{
set$orig_args$args;
set$args"a=3&b=4";
echo"originalargs:$orig_args";
echo"args:$args";
}
HerewefirstsavetheoriginalURLquerystringintoourownvariable$orig_args,thenmodifythecurrentquerystringbyoverridingthe$argsvariable,andfinallyoutputthevariables$orig_argsand$args,respectively,withtheechodirective.Let'stestitlikethis:
$curl'http://localhost:8080/test'
originalargs:
args:a=3&b=4
$curl'http://localhost:8080/test?a=0&b=1&c=2'
originalargs:a=0&b=1&c=2
args:a=3&b=4
Inthefirsttest,wedidnotprovideanyURLquerystring,hencetheemptyoutputforthe$orig_argsvariable.Andinbothtests,thecurrentquerystringwasforciblyoverriddentothenewvaluea=3&b=4,regardlessofthepresenceofaquerystringintheoriginalrequest.
Itshouldbenotedthatthe$argsvariableherenolongerownsavaluecontainerasuservariables,justlike$arg_XXX.Whenreading$args,Nginxwillexecuteaspecialpieceofcode,fetchingdatafromaparticularplacewheretheNginxcorestorestheURLquerystringforthecurrentrequest.Ontheotherhand,whenweoverwrite$args,Nginxwillexecuteanotherspecialpieceofcode,storingnewvalueintothesameplaceinthecore.OtherpartsofNginxalsoreadthesameplacewheneverthequerystringisneeded,soourmodificationto$argswillimmediatelyaffectalltheotherparts'functionalitylateron.Let'sseeanexampleforthis:
location/test{
set$orig_a$arg_a;
set$args"a=5";
echo"originala:$orig_a";
echo"a:$arg_a";
}
Herewefirstsavethevalueofthebuilt-invaraible$arg_a,thevalueoftheoriginalrequest'sURLargumenta,intoouruservariable$orig_a,thenchangetheURLquerystringtoa=5byassigningthenewvaluetothebuilt-invariable$args,andfinallyoutputthevariables$orig_aand$arg_a,respectively.Becausemodificationsto$argseffectivelychangetheURLquerystringofthecurrentrequestforthewholeserver,thevalueofthebuilt-invariable$arg_XXXshouldalsochangeaccordingly.Thetestresultverifiesthis:
$curl'http://localhost:8080/test?a=3'
originala:3
a:5
Wecanseethattheinitialvalueof$arg_ais3sincetheURLquerystringoftheoriginalrequestisa=3.Butthefinalvalueof$arg_aautomaticallybecomes5afterwemodify$argswiththevaluea=5.
Belowisanotherexampletodemonstratethatassignmentsto$argsalsoaffecttheHTTPproxymodulengx_proxy.
server{
listen8080;
location/test{
set$args"foo=1&bar=2";
proxy_passhttp://127.0.0.1:8081/args;
}
}
server{
listen8081;
location/args{
echo"args:$args";
}
}
Twovirtualserversaredefinedhereinthehttpconfigurationblock(omittedforbrevity).
Thefirstvirtualserverislisteningatthelocalport8080.Its/testlocationfirstupdatesthecurrentURLquerystringtothevaluefoo=1&bar=2bywritingto$args,thensetsupanHTTPreverseproxyviatheproxy_passdirectiveofthengx_proxymodule,targetingtheHTTPservice/argsonthelocalport8081.Bydefaultthengx_proxymoduleautomaticallyforwardsthecurrentURLquerystringtotheremoteHTTPservice.
The"remoteHTTPservice"onthelocalport8081isprovidedbythesecondvirtualserverdefinedbyourselves,whereweoutputthecurrentURLquerystringviatheechodirectiveinlocation/args.Bydoingthis,wecaninvestigatetheactualURLquerystringforwardedbythengx_proxymodulefromthefirstvirtualserver.
Let'saccessthe/testinterfaceexposedbythefirstvirtualserver.
$curl'http://localhost:8080/test?blah=7'
args:foo=1&bar=2
WecanseethattheURLquerystringisfirstrewrittentofoo=1&bar=2eventhoughtheoriginalrequesttakesthevalueblah=7,thenitisforwardedtothe/argsinterfaceofthesecondvirtualserverviatheproxy_passdirective,andfinallyitsvalueisoutputtotheclient.
Tosummarize,theassignmentto$argsalsosuccessfullyinfluencesthebehaviorofthengx_proxymodule.
Variable"GetHandlers"and"SetHandlers"
Wehavealreadylearnedinprevioussectionsthatwhenreadingthebuilt-invariable$args,Nginxexecutesaspecialpieceofcodetoobtainavalueon-the-flyandwhenwritingtothisvariable,Nginxexecutesanotherspecialpieceofcodetopropagatethechange.InNginx'sterminology,thespecialcodeexecutedforreadingthevariableiscalled"gethandler"andthecodeforwritingtothevariableiscalled"sethandler".DifferentNginxmodulesusuallypreparedifferent"gethandlers"and"sethandlers"fortheirownvariables,whicheffectivelyputmagicintothesevariables'behavior.
Suchtechniquesarenotuncommoninthecomputingworld.Forexample,inobject-orientedprogramming(OOP),theclassdesignerusuallydoesnotexposethemembervariableoftheclassdirectlytotheuserprogrammer,butinsteadprovidestwomethodsforreadingfromandwritingtothemembervariable,respectively.Suchclassmethodsareoftencalled"accessors".BelowisanexampleintheC++programminglanguage:
#include<string>
usingnamespacestd;
classPerson{
public:
conststringget_name(){
returnm_name;
}
voidset_name(conststringname){
m_name=name;
}
private:
stringm_name;
};
InthisC++classPerson,weprovidetwopublicmethods,get_nameandset_name,toserveasthe"accessors"fortheprivatemembervariablem_name.
Thebenefitsofsuchdesignareobvious.Theclassdesignercanexecutearbitrarycodeinthe"accessors",toimplementanyextrabusinesslogicorusefulsideeffects,likeautomaticallyupdatingothermembervariablesdependingonthecurrentmember,orupdatingthecorrespondingfieldinadatabaseassociatedwiththecurrentobject.Forthelattercase,itispossiblethatthemembervariabledoesnotexistatall,orthatthemembervariablejustservesasadatacachetomitigatethepressureontheback-enddatabase.
Correspondingtotheconceptof"accessors"inOOP,Nginxvariablesalsosupportbindingcustom"gethandlers"and"sethandlers".Additionally,notallNginxvariablesownacontainertoholdvalues.Somevariableswithoutacontainerjustbehavelikeamagicalcontainerbymeansofitsfancy"gethandler"and"sethandler".Infact,whenavariableisbeingcreatedat"configuretime",thecreatingNginxmodulemustmakeadecisiononwhethertoallocateavaluecontainerforitandwhethertoattachacustom"gethandler"and/ora"sethandler"toit.
Thosevariablesowningavaluecontainerarecalled"indexedvariables"inNginx'sterminology.Otherwise,theyaresaidtobenotindexed.
Wealreadyknowthatthe"variablegroups"like$arg_XXXdiscussedinearliersectionsdonothaveavaluecontainerandthusarenotindexed.Whenreading$arg_XXX,itisits"gethandler"atwork,thatis,its"gethandler"scansthecurrentURLquerystringon-the-fly,extractingthevalueofthespecifiedURLargument.Manybeginnersmisunderstandtheway$arg_XXXisimplemented;theyassumethatNginxwillparsealltheURLargumentsinadvanceandpreparethevaluesforallthosenon-empty$arg_XXXvariablesbeforetheyareactuallyread.Thisisnottrue,however.NginxnevertriestoparsealltheURLargumentsbeforehand,butratherscansthewholeURLquerystringforaparticularargumentina"gethandler"everytimethatargumentisrequestedbyreadingthecorresponding$arg_XXXvariable.Similarly,whenreadingthebuilt-invariable$cookie_XXX,its"gethandler"justscanstheCookierequestheadersforthecookienamespecified.
NginxVariables(04)ValueContainersforCaching&ngx_map
SomeNginxvariableschoosetousetheirvaluecontainersasadatacachewhenthe"gethandler"isconfigured.Inthissetting,the"gethandler"isrunonlyonce,i.e.,atthefirsttimethevariableisread,whichreducesoverheadwhenthevariableisreadmultipletimesduringitslifetime.Let'sseeanexampleforthis.
map$args$foo{
default0;
debug1;
}
server{
listen8080;
location/test{
set$orig_foo$foo;
set$argsdebug;
echo"originalfoo:$orig_foo";
echo"foo:$foo";
}
}
Hereweusethemapdirectivefromthestandardmodulengx_mapforthefirsttime,whichdeservessomeintroduction.Thewordmapheremeansmappingorcorrespondence.Forexample,functionsinMathsareakindof"mapping".AndNginx'smapdirectiveisusedtodefinea"mapping"relationshipbetweentwoNginxvariables,orinotherwords,"functionrelationship".Backtothisexample,weusethemapdirectivetodefinethe"mapping"relationshipbetweenuservariable$fooandbuilt-invariable$args.WhenusingtheMathfunctionnotation,y=f(x),our$argsvariableiseffectivelythe"independentvariable",x,while$fooisthe"dependentvariable",y.Thatis,thevalueof$foodependsonthevalueof$args,orrather,wemapthevalueof$argsontothe$foovariable(insomeway).
Nowlet'slookattheexactmappingruledefinedbythemapdirectiveinthisexample.
map$args$foo{
default0;
debug1;
}
Thefirstlinewithinthecurlybracesisaspecialrulecondition,thatis,thisconditionholdsifandonlyifotherconditionsallfail.Whenthis"default"conditionholds,the"dependentvariable"$fooisassignedbythevalue0.Thesecondlinewithinthecurlybracesmeansthatthe"dependentvariable"$fooisassignedbythevalue1ifthe"independentvariable"$argsmatchesthestringvaluedebug.Combiningthesetwolines,weobtainthefollowingcompletemappingrule:ifthevalueof$argsisdebug,variable$foogetsthevalue1;otherwise$foogetsthevalue0.Soessentially,thisisaconditionalassignmenttothevariable$foo.
Nowthatweunderstandwhatthemapdirectivedoes,let'slookatthedefinitionoflocation/test.Wefirstsavethevalueof$foointoanotheruservariable$orig_foo,thenoverwritethevalueof$argstodebug,andfinallyoutputthevaluesof$orig_fooand$foo,respectively.
Intuitively,afterweoverwritethevalueof$argstodebug,thevalueof$fooshouldautomaticallygetadjustedto1accordingtothemappingruledefinedearlier,regardlessoftheoriginalvalueof$foo.Butthetestresultsuggeststheotherwayaround.
$curl'http://localhost:8080/test'
originalfoo:0
foo:0
Thefirstoutputlineindicatesthatthevalueof$orig_foois0,whichisexactlywhatweexpected:theoriginalrequestdoesnottakeaURLquerystring,sotheinitialvalueof$argsisempty,leadingtothe0initialvalueof$foo,accordingtothe"default"conditioninourmappingrule.
Butsurprisingly,thesecondoutputlineindicatesthatthefinalvalueof$fooisstill0,evenafterweoverwrite$argstothevaluedebug.Thisapparentlyviolatesourmappingrulebecausewhen$argstakesthevaluedebug,thevalueof$fooshouldreallybe1.Sowhatishappeninghere?
Actuallythereasonisprettysimple:whenthefirsttimevariable$fooisread,itsvaluecomputedbyngx_map's"gethandler"iscachedinitsvaluecontainer.WealreadylearnedearlierthatNginxmodulesmaychoosetousethevaluecontainerofthevariablecreatedbythemselvesasadatacacheforits"gethandler".Obviously,thengx_mapmoduleconsidersthemappingcomputationbetweenvariablesexpensiveenoughandcachestheresultautomatically,sothatthenexttimethesamevariableisreadwithinthelifetimeofthecurrentrequest,Nginxcanjustreturnthecachedresultwithoutinvokingthe"gethandler"again.
Toverifythisfurther,wecantryspecifyingtheURLquerystringasdebugintheoriginalrequest.
$curl'http://localhost:8080/test?debug'
originalfoo:1
foo:1
Itcanbeseenthatthevalueof$orig_foobecomes1,complyingwithourmappingrule.Andsubsequentreadingsof$fooalwaysyieldthesamecachedresult,1,regardlessofthenewvalueof$argslateron.
Themapdirectiveisactuallyauniqueexample,becauseitnotonlyregistersa"gethandler"fortheuservariable,butalsoallowstheusertodefinethecomputingruleinthe"gethandler"directlyintheNginxconfigurationfile.Ofcourse,therulethatcanbedefinedhereislimitedtosimplemappingrelationswithanothervariable.Meanwhile,itmustbemadeclearthatnotallthevariablesusinga"gethandler"willcachetheresult.Forinstance,wehavealreadyseenearlierthatthe$arg_XXXvariabledoesnotuseitsvaluecontaineratall.
Similartothengx_mapmodule,thestandardmodulengx_geothatweencounteredearlieralsoenablesvaluecachingforthevariablescreatedbyitsgeodirective.
ASideNoteforUseContextsofDirectives
Inthepreviousexample,weshouldalsonotethatthemapdirectiveisputoutsidetheserverconfigurationblock,thatis,itisdefineddirectlywithintheoutermosthttpconfigurationblock.Somereadersmaybecuriousaboutthissetting,sinceweonlyuseitinlocation/testafterall.Ifwetryputtingthemapstatementwithinthelocationblock,however,wewillgetthefollowingerrorwhilestartingNginx:
[emerg]"map"directiveisnotallowedherein...
Soitisexplicitlyprohibited.Infact,itisonlyallowedtousethemapdirectiveinthehttpblock.Everyconfiguredirectivedoeshaveapre-definedsetofusecontextsintheconfigurationfile.Whenindoubt,alwaysrefertothecorrespondingdocumentationfortheexactusecontextsofaparticulardirective.
LazyEvaluationofVariableValues
ManyNginxfreshmenwouldworrythattheuseofthemapdirectivewithintheglobalscope(i.e.,thehttpblock)willleadtounnecessaryvariablevaluecomputationandassignmentforallthelocationsinallthevirtualserversevenifonlyonelocationblockactuallyusesit.Fortunately,thisisnotwhatishappeninghere.Wehavealreadylearnedhowthemapdirectiveworks.Itisthe"gethandler"(registeredbythengx_mapmodule)thatperformsthevaluecomputationandrelatedassignment.Andthe"gethandler"willnotrunatallunlessthecorrespondinguservariableisactuallybeingread.Therefore,forthoserequeststhatneveraccessthatvariable,therecannotbeany(useless)computationinvolved.
Thetechniquethatpostponesthevaluecomputationofftothepointwherethevalueisactuallyneedediscalled"lazyevaluation"inthecomputingworld.Programminglanguagesnativelyoffering"lazyevaluation"isnotverycommonthough.ThemostfamousexampleistheHaskellprogramminglanguage,wherelazyevaluationisthedefaultsemantics.Incontrastwith"lazyevaluation",itismuchmorecommontosee"eagerevaluation".Weareluckytoseeexamplesoflazyevaluationhereinthengx_mapmodule,butthe"eagerevaluation"semanticsisalsomuchmorecommonintheNginxworld.Considerthefollowingsetstatementthatcannotbesimpler:
set$b"$a,$a";
Whenrunningthesetdirective,Nginxeagerlycomputesandassignsthenewvalueforthevariable$bwithoutpostponingtothepointwhen$bisactuallyreadlateron.Similarly,theset_unescape_uridirectivealsoevaluateseagerly.
NginxVariables(05)VariablesinSubrequests
ADetourtoSubrequests
Wehaveseenearlierthatthelifetimeofvariablecontainersisboundtotherequest,butIownyouaformaldefinitionof"requests"there.Youmighthaveassumedthatthe"requests"inthatcontextarejustthoseHTTPrequestsinitiatedfromtheclientside.Infact,therearetwokindsof"requests"intheNginxworld.Oneiscalled"mainrequests",andtheotheriscalled"subrequests".
MainrequestsarethoseinitiatedexternallybyHTTPclients.Alltheexamplesthatwehaveseensofarinvolvemainrequestsonly,includingthosedoing"internalredirections"viatheecho_execorrewritedirective.
WhereassubrequestsareaspecialkindofrequestsinitiatedfromwithintheNginxcore.ButpleasedonotconfusesubrequestswiththoseHTTPrequestscreatedbythengx_proxymodules!SubrequestsmaylookverymuchlikeanHTTPrequestinappearance,theirimplementation,however,hasnothingtodowithneithertheHTTPprotocolnoranykindofsocketcommunication.Asubrequestisanabstractinvocationfordecomposingthetaskofthemainrequestintosmaller"internalrequests"thatcanbeservedindependentlybymultipledifferentlocationblocks,eitherinseriesorinparallel."Subrequests"canalsoberecursive:anysubrequestcaninitiatemoresub-subrequests,targetingotherlocationblocksoreventhecurrentlocationitself.AccordingtoNginx'sterminology,ifrequestAinitiatesasubrequestB,thenAiscalledthe"parentrequest"ofB.ItisworthmentioningthattheApachewebserveralsohastheconceptofsubrequestsforlong,soreaderscomingfromthatworldshouldbenostrangertothis.
Let'scheckoutanexampleusingsubrequests:
location/main{
echo_location/foo;
echo_location/bar;
}
location/foo{
echofoo;
}
location/bar{
echobar;
}
Hereinlocation/main,weusetheecho_locationdirectivefromthengx_echomoduletoinitiatetwoGET-typedsubrequeststargeting/fooand/bar,respectively.Thesubrequestsinitiatedbyecho_locationarealwaysrunningsequentiallyaccordingtotheirliteralorderintheconfigurationfile.Therefore,thesecond/barrequestwillnotbefireduntilthefirst/foorequestcompletesprocessing.Theresponsebodyofthesetwosubrequestsgetconcatenatedtogetheraccordingtotheirrunningorder,toformthefinalresponsebodyoftheirparentrequest(for/main):
$curl'http://localhost:8080/main'
foo
bar
Itshouldbenotedthatthecommunicationoflocationblocksviasubrequestsislimitedwithinthesameserverblock(i.e.,thesamevirtualserverconfiguration),sowhentheNginxcoreprocessesasubrequest,itjustcallsafewCfunctionsbehindthescene,withoutdoinganykindofnetworkorUNIXdomainsocketcommunication.Forthisreason,subrequestsareextremelyefficient.
IndependentVariableContainersinSubrequests
BacktoourearlierdiscussionforthelifetimeofNginxvariablecontainers,nowwecanstillstatethatthelifetimeisboundtothecurrentrequest,andeveryrequestdoeshaveitsowncopyofallthevariablecontainers.Itisjustthatthe"request"herecanbeeitheramainrequest,orasubrequest.Variableswiththesamenamebetweenaparentrequestandasubrequestwillgenerallynotinterferewitheachother.Let'sdoasmallexperimenttoconfirmthis:
location/main{
set$varmain;
echo_location/foo;
echo_location/bar;
echo"main:$var";
}
location/foo{
set$varfoo;
echo"foo:$var";
}
location/bar{
set$varbar;
echo"bar:$var";
}
Inthissample,weassigndifferentvaluestothevariable$varinthreelocationblocks,/main,/foo,and/bar,andoutputthevalueof$varinalltheselocations.Inparticular,weintentionallyoutputthevalueof$varinlocation/mainaftercallingthetwosubrequests,soifvaluechangesof$varinthesubrequestscanaffecttheirparentrequest,weshouldseeanewvalueoutputinlocation/main.Theresultofrequesting/mainisasfollows:
$curl'http://localhost:8080/main'
foo:foo
bar:bar
main:main
Apparently,theassignmentstovariable$varinthosetwosubrequestsdonotaffectthemainrequest/mainatall.Thissuccessfullyverifiesthatboththemainrequestanditssubrequestsdoowndifferentcopiesofvariablecontainers.
SharedVariableContainersamongRequests
Unfortunately,subrequestsinitiatedbycertainNginxmodulesdosharevariablecontainerswiththeirparentrequests,likethoseinitiatedbythe3rd-partymodulengx_auth_request.Belowissuchanexample:
location/main{
set$varmain;
auth_request/sub;
echo"main:$var";
}
location/sub{
set$varsub;
echo"sub:$var";
}
Hereinlocation/main,wefirstassigntheinitialvaluemaintovariable$var,thenfireasubrequestto/subviatheauth_requestdirectivefromthengx_auth_requestmodule,andfinallyoutputthevalueof$var.Notethatinlocation/subweintentionallyoverwritethevalueof$vartosub.Whenaccessing/main,weget
$curl'http://localhost:8080/main'
main:sub
Obviously,thevaluechangeof$varinthesubrequestto/subdoesaffectthemainrequestto/main.Thusthevariablecontainerof$varisindeedsharedbetweenthemainrequestandthesubrequestcreatedbythengx_auth_requestmodule.
Forthepreviousexample,somereadersmightask:"whydoesn'ttheresponsebodyofthesubrequestappearinthefinaloutput?"Theanswerissimple:itisjustbecausetheauth_requestdirectivediscardstheresponsebodyofthesubrequestitmanages,andonlycheckstheresponsestatuscodeofthesubrequest.Whenthestatuscodelooksgood,like200,auth_requestwilljustallowNginxcontinueprocessingthemainrequest;otherwiseitwillimmediatelyabortthemainrequestbyreturninga403errorpage,forexample.Inourexample,thesubrequestto/subjustreturna200responseimplicitlycreatedbytheechodirectiveinlocation/sub.
Eventhoughsharingvariablecontainersamongthemainrequestandallitssubrequestscouldmakebidirectionaldataexchangeeasier,itcouldalsoleadtounexpectedsubtleissuesthatarehardtodebuginreal-worldconfigurations.Becauseusersoftenforgetthatavariablewiththesamenameisactuallyusedinsomedeeplyembeddedsubrequestandjustuseitforsomethingelseinthemainrequest,thisvariablecouldgetunexpectedlymodifiedduringprocessing.Suchbadsideeffectsmakemany3rd-partymoduleslikengx_echo,ngx_luaandngx_srcachechoosetodisablethevariablesharingbehaviorforsubrequestsbydefault.
NginxVariables(06)Built-inVariablesinSubrequests
TherearesomesubtletiesinvolvedinusingNginxbuilt-invariablesinthecontextofasubrequest.Wewilldiscussthedetailsinthissection.
Built-inVariablesSensitivetotheSubrequestContext
Wealreadyknowthatmostbuilt-invariablesarenotsimplevaluecontainers.Theybehavedifferentlythanuservariablesbyregistering"gethandlers"and/or"sethandlers".Evenwhentheydoownavaluecontainer,theyusuallyjustusethecontainerasaresultcachefortheir"gethandlers".The$argsvariablewediscussedearlier,forexample,justusesits"gethandler"toreturntheURLquerystringforthecurrentrequest.Thecurrentrequestherecanalsobeasubrequest,sowhenreading$argsinasubrequest,its"gethandler"shouldnaturallyreturnthequerystringforthesubrequest.Let'sseesuchanexample:
location/main{
echo"mainargs:$args";
echo_location/sub"a=1&b=2";
}
location/sub{
echo"subargs:$args";
}
Hereinthe/maininterface,wefirstechooutthevalueof$argsforthecurrentrequest,andthenuseecho_locationtoinitiateasubrequestto/sub.Itshouldbenotedthatherewegiveasecondargumenttotheecho_locationdirective,tospecifytheURLquerystringforthesubrequestbeingfired(thefirstargumentistheURIforthesubrequest,aswealreadyknow).Finally,wedefinethe/subinterfaceandprintoutthevalueof$argsinthere.Queryingthe/maininterfacegives
$curl'http://localhost:8080/main?c=3'
mainargs:c=3
subargs:a=1&b=2
Itisclearthatwhen$argsisreadinthemainrequest(to/main),itsvalueistheURLquerystringofthemainrequest;whereaswheninthesubrequest(to/foo),itisthequerystringofthesubrequest,a=1&b=2.Thisbehaviorindeedmatchesourintuition.
Justlike$args,whenthebuilt-invariable$uriisusedinasubrequest,its"gethandler"alsoreturnsthe(decoded)URIofthecurrentsubrequest:
location/main{
echo"mainuri:$uri";
echo_location/sub;
}
location/sub{
echo"suburi:$uri";
}
Belowistheresultofquerying/main:
$curl'http://localhost:8080/main'
mainuri:/main
suburi:/sub
Theoutputiswhatwewouldexpect.
Built-inVariablesforMainRequestsOnly
Unfortunately,notallbuilt-invariablesaresensitivetothecontextofsubrequests.Severalbuilt-invariablesalwaysactonthemainrequestevenwhentheyareusedinasubrequest.Thebuilt-invariable$request_methodissuchanexception.
Whenever$request_methodisread,wealwaysgettherequestmethodname(suchasGETandPOST)forthemainrequest,nomatterwhetherthecurrentrequestisasubrequestornot.Let'stestitout:
location/main{
echo"mainmethod:$request_method";
echo_location/sub;
}
location/sub{
echo"submethod:$request_method";
}
Inthisexample,the/mainand/subinterfacesbothoutputthevalueof$request_method.Meanwhile,weinitiateaGETsubrequestto/subviatheecho_locationdirectivein/main.Nowlet'sdoaPOSTrequestto/main:
$curl--datahello'http://localhost:8080/main'
mainmethod:POST
submethod:POST
Hereweusethe--dataoptionofthecurlutilitytospecifyourPOSTrequestbody,alsothisoptionmakescurlusethePOSTmethodfortherequest.Thetestresultturnsoutaswepredicted:thevariable$request_methodisevaluatedtothemainrequest'smethodname,POST,despiteitsuseinaGETsubrequest.
Somereadersmightchallengeourconclusionherebypointingoutthatwedidnotruleoutthepossibilitythatthevalueof$request_methodgotcachedatitsfirstreadinginthemainrequestandwhatwewereseeinginthesubrequestwasactuallythecachedvaluethatwasevaluatedearlierinthemainrequest.Thisconcernis
unnecessary,however,becausewehavealsolearnedthatthevariablecontainerrequiredbydatacaching(ifany)isalwaysboundtothecurrentrequest,alsothesubrequestsinitiatedbythengx_echomodulealwaysdisablevariablecontainersharingwiththeirparentrequests.Backtothepreviousexample,evenifthebuilt-invariable$request_methodinthemainrequestusedthevaluecontainerasthedatacache(actuallyitdoesnot),itcannotaffectthesubrequestbyanymeans.
Tofurtheraddresstheconcernofthesereaders,let'sslightlymodifythepreviousexamplebyputtingtheechostatementfor$request_methodin/mainaftertheecho_locationdirectivethatrunsthesubrequest:
location/main{
echo_location/sub;
echo"mainmethod:$request_method";
}
location/sub{
echo"submethod:$request_method";
}
Let'stestitagain:
$curl--datahello'http://localhost:8080/main'
submethod:POST
mainmethod:POST
Nochangeintheoutputcanbeobserved,exceptthatthetwooutputlinesreversedtheorder(sinceweexchangetheorderofthosetwongx_echomodule'sdirectives).
Consequently,wecannotobtainthemethodnameofasubrequestbyreadingthe$request_methodvariable.Thisisacommonpitfallforfreshmenwhendealingwithmethodnamesofsubrequests.Toovercomethislimitation,weneedtoturntothebuilt-invariable$echo_request_methodprovidedbythengx_echomodule:
location/main{
echo"mainmethod:$echo_request_method";
echo_location/sub;
}
location/sub{
echo"submethod:$echo_request_method";
}
Wearefinallygettingwhatwewant:
$curl--datahello'http://localhost:8080/main'
mainmethod:POST
submethod:GET
Nowwithinthesubrequest,wegetitsownmethodname,GET,asexpected,andthemainrequestmethodremainsPOST.
Similarto$request_method,thebuilt-invariable$request_urialsoalwaysreturnsthe(non-decoded)URLforthemainrequest.Thisismoreunderstandable,however,becausesubrequestsareessentiallyfakedrequestsinsideNginx,whichdonotreallytakeanon-decodedrawURL.
VariableContainerSharingandValueCachingTogether
Intheprevioussection,someofthereaderswereworriedaboutthecasethatvariablecontainersharinginsubrequestsandvaluecachingforvariable's"gethandlers"wereworkingtogether.Ifitwereindeedthecase,thenitwouldbeanightmarebecauseitwouldbereallyreallyhardtopredictwhatisgoingonbyjustlookingattheconfigurationfile.Inprevioussections,wealreadylearnedthatthesubrequestsinitiatedbythengx_auth_requestmodulearesharingthesamevariablecontainerswiththeirparents,sowecanmaliciouslyconstructsuchahorribleexample:
map$uri$tag{
default0;
/main1;
/sub2;
}
server{
listen8080;
location/main{
auth_request/sub;
echo"maintag:$tag";
}
location/sub{
echo"subtag:$tag";
}
}
Hereweuseouroldfriend,themapdirective,tomapthevalueofthebuilt-invariable$uritoouruservariable$tag.When$uritakesthevalue/main,thevalue1isassignedto$tag;when$uritakesthevalue/sub,thevalue2isassignedinsteadto$tag;underalltheotherconditions,0isassigned.Next,in/main,wefirstinitiateasubrequestto/subbyusingtheauth_requestdirective,andthenoutputthevalueof$tag.Andwithin/sub,wedirectlyoutputthevalueof$tag.Guesswhatwewillgetwhenweaccess/main?
$curl'http://localhost:8080/main'
maintag:2
Ouch!Didn'twemapthevalue/mainto1?Whytheactualoutputfor/mainisthevalue,2,for/sub?Whatisgoingonhere?
Actuallyitworkedlikethis:our$tagvariablewasfirstreadinthesubrequestto/sub,andthe"gethandler"registeredbymapcomputedthevalue2for$taginthatcontext(because$uriwas/subinthesubrequest)andthevalue2gotcachedinthevaluecontainerof$tagfromthenon.Becausetheparentrequestsharedthesamecontainerasthesubrequestcreatedbyauth_request,whentheparentrequestread$taglater(afterthesubrequestwasfinished),thecachedvalue2wasdirectlyreturned!Suchresultscanindeedbeverysurprisingatfirstglance.
Fromthisexample,wecanconcludeagainthatitcanhardlybeagoodideatoenablevariablecontainersharinginsubrequests.
NginxVariables(07)SpecialValue"Invalid"and"NotFound"
WehavementionedthatthevaluesofNginxvariablescanonlybeofonesingletype,thatis,thestringtype,butvariablescouldalsohavenomeaningfulvaluesatall.Variableswithoutanymeaningfulvaluesstilltakeaspecialvaluethough.Therearetwopossiblespecialvalues:"invalid"and"notfound".
Forexample,whenauservariable$fooiscreatedbutnotassignedyet,$footakesthespecialvalueof"invalid".AndwhenthecurrentURLquerystringdoesnothavetheXXXargumentatall,thebuilt-invariable$arg_XXXtakesthespecialvalueof"notfound".
Both"invalid"and"notfound"arespecialvalues,completelydifferentfromanemptystringvalue("").Thisisverysimilartothosedistinctspecialvaluesinsomedynamicprograminglanguages,likeundefinPerl,nilinLua,andnullinJavaScript.
Wehaveseenearlierthatanuninitializedvariableisevaluatedtoanemptystringwhenusedinaninterpolatedstring,itsrealvalue,however,isnotanemptystringatall.Itisthe"gethandler"registeredbythesetdirectivethatautomaticallyconvertsthe"invalid"specialvalueintoanemptystring.Toverifythis,let'sreturntotheexamplewehavediscussedbefore:
location/foo{
echo"foo=[$foo]";
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
Whenaccessing/foo,theuservariable$fooisuninitializedwhenusedintheinterpolatedstringfortheechodirective.Theoutputshowsthatthevariableisevaluatedtoanemptystring:
$curl'http://localhost:8080/foo'
foo=[]
Fromtheoutput,theuninitialized$foovariablebehavesjustliketakinganemptystringvalue.Butcarefulreadersshouldhavealreadynoticedthat,fortherequestabove,thereisawarningintheNginxerrorlogfile(whichislogs/error.logbydefault):
[warn]5765#0:*1usinguninitialized"foo"variable,...
Whoonearthgeneratesthiswarning?Theansweristhe"gethandler"of$foo,registeredbythesetdirective.When$fooisread,Nginxfirstchecksthevalueinitscontainerbutseesthe"invalid"specialvalue,thenNginxdecidestocontinuerunning$foo's"gethandler",whichfirstprintsthewarning(asshownabove)andthenreturnsanemptystringvalue,whichthereaftergetscachedin$foo'svaluecontainer.
Carefulreadersshouldhaveidentifiedthatthisprocessforuservariablesisexactlythesameasthemechanismwediscussedearlierforbuilt-invariablesinvolving"gethandlers"andresultcachinginvaluecontainers.Yes,itisthesamemechanisminaction.Itisalsoworthnotingthatonlythe"invalid"specialvaluewilltriggerthe"gethandler"invocationintheNginxcorewhile"notfound"willnot.
Thewarningmessageaboveusuallyindicatesatypointhevariablenameormisuseofuninitializedvariables,notnecessarilyinthecontextofaninterpolatedstring.Becauseoftheexistenceofvaluecachinginthevariablecontainer,thiswarningwillnotgetprintedmultipletimesinthelifetimeofthecurrentrequest.Also,thengx_rewritemoduleprovidestheuninitialized_variable_warndirectivefordisablingthiswarningaltogether.
TestingSpecialValuesofNginxVariablesinLua
Aswehavejustmentioned,thebuilt-invariable$arg_XXXtakesthespecialvalue"notfound"whentheURLargumentXXXdoesnotexist,butunfortunately,itisnoteasytodistinguishitfromtheemptystringvaluedirectlyintheNginxconfigurationfile,forexample:
location/test{
echo"name:[$arg_name]";
}
HereweintentionallyomittheURLargumentnameinourrequest:
$curl'http://localhost:8080/test'
name:[]
Wecanseethatwearestillgettinganemptystringvalue,becausethistimeitistheNginx"scriptengine"thatautomaticallyconvertsthe"notfound"specialvaluetoanemptystringwhenperformingvariableinterpolation.
Thenhowcanwetestthespecialvalue"notfound"?Orinotherwords,howcanwedistinguishitfromnormalemptystringvalues?Obviously,inthefollowingexample,theURLargumentnamedoestakeanordinaryvalue,whichisatrueemptystring:
$curl'http://localhost:8080/test?name='
name:[]
Butwecannotreallydifferentiatethisfromtheearliercasethatdoesnotmentionthenameargumentatall.
Luckily,wecaneasilyachievethisinLuabymeansofthe3rd-partymodulengx_lua.Pleaselookatthefollowingexample:
location/test{
content_by_lua'
ifngx.var.arg_name==nilthen
ngx.say("name:missing")
else
ngx.say("name:[",ngx.var.arg_name,"]")
end
';
}
Thisexampleisveryclosetothepreviousoneintermsoffunctionality.Weusethecontent_by_luadirectivefromthengx_luamoduletoembedasmallpieceofourownLuacodetotestagainstthespecialvalueoftheNginxvariable$arg_name.When$arg_nametakesaspecialvalue(either"notfound"or"invalid"),wewillgetthefollowingoutputwhenrequesting/foo:
$curl'http://localhost:8080/test'
name:missing
Thisisourfirsttimemeetingthengx_luamodule,whichdeservesabriefintroduction.ThismoduleembedstheLualanguageinterpreter(orLuaJIT'sJust-in-Timecompiler)intotheNginxcore,toallowNginxusersdirectlyruntheirownLuaprogramsinsidetheserver.TheusercanchoosetoinsertherLuacodeintodifferentrunningphasesoftheserver,tofulfilldifferentrequirements.SuchLuacodeareeitherspecifieddirectlyasliteralstringsintheNginxconfigurationfile,orresideinexternal.luasourcefiles(orLuabinarybytecodefiles)whosepathsarespecifiedintheNginxconfiguration.
Backtoourexample,wecannotdirectlywritesomethinglike$arg_nameinourLuacode.Instead,wereferenceNginxvariablesinLuabymeansofthengx.varAPIprovidedbythengx_luamodule.Forexample,toreferencetheNginxvariable$VARIABLEinLua,wejustwritengx.var.VARIABLE.WhentheNginxvariable$arg_nametakesthespecialvalue"notfound"(or"invalid"),ngx.var.arg_nameisevaluatedtothenilvalueintheLuaworld.ItshouldalsobenotingthatweusetheLuafunctionngx.saytoprintouttheresponsebodycontents,whichisfunctionallyequivalenttotheechodirectivewearealreadyveryfamiliarwith.
IfweprovideanameURIargumentthattakesanemptyvalueintherequest,theoutputisnowverydifferent:
$curl'http://localhost:8080/test?name='
name:[]
Inthistest,thevalueoftheNginxvariable$arg_nameisatrueemptystring,neither"notfound"nor"invalid".SoinLua,theexpressionngx.var.arg_nameevaluatestotheLuaemptystring(""),clearlydistinguishedfromtheLuanilvalueintheprevioustest.
Thisdifferentiationisimportantincertainapplicationscenarios.Forinstance,somewebserviceshavetodecidewhethertouseacolumnvaluetofilterthedatasetbycheckingtheexistenceofthecorrespondingURIargument.Fortheseserives,whenthenameURIargumentisabsent,thewholedatasetarejustreturned;whenthenameargumenttakesanemptyvalue,however,onlythoserecordsthattakeanemptyvaluearereturned.
Itisworthmentioningafewlimitationsinthestandard$arg_XXXvariable.Considerusingthefollowingrequesttotest/testinourpreviousexampleusingLua:
$curl'http://localhost:8080/test?name'
name:missing
Nowthe$arg_namevariablestillreadsthe"notfound"specialvalue,whichisapparentlycounter-intuitive.Additionally,whenmultipleURIargumentswiththesamenamearespecifiedintherequest,$arg_XXXjustreturnsthefirstvalueoftheargument,discardingothervaluessilently:
$curl'http://localhost:8080/test?name=Tom&name=Jim&name=Bob'
name:[Tom]
Tosolvetheseproblems,wecanusetheLuafunctionngx.req.get_uri_argsprovidedbythengx_luamoduleinstead.
NginxVariables(08)In(02)wementionedthatanothercategoryofbuiltinvariables$cookie_XXXarelike$arg_XXX.SimilarlywhenthereexistnocookienamedXXX,itscorrespondingNginxvariable$cookie_XXXhasnon-value"notfound".
location/test{
content_by_lua'
ifngx.var.cookie_user==nilthen
ngx.say("cookieuser:missing")
else
ngx.say("cookieuser:[",ngx.var.cookie_user,"]")
end
';
}
Thecurlutilityoffersthe--cookiename=valueoption,whichdesignatesname=valueasacookieofitsrequest(byaddingtheCookieheader).Let'stestafewcasescontainingcookies.
$curl--cookieuser=agentzh'http://localhost:8080/test'
cookieuser:[agentzh]
$curl--cookieuser='http://localhost:8080/test'
cookieuser:[]
$curl'http://localhost:8080/test'
cookieuser:missing
Asexpected,whencookieuserdoesnotexist,Luavariablengx.var.cookie_userisnil.Sowehavesuccessfullydistinguishedthecasewithemptystringandthecasewithnon-value.
Aniceadd-onwithmodulengx_luaiswhenluareferencesanundeclaredvariableofNginx,thevariableisnilandNginxwillnotabortsitloadingasbefore.
location/test{
content_by_lua'
ngx.say("$blah=",ngx.var.blah)
';
}
Uservariable$blahisneverdeclaredintheNginxconfigurationnginx.conf,butitisreferencedasngx.var.blahinLuacode.Nginxcanbestartedstill,becausewhenNginxloadsitsconfiguration,Luacodeisonlycompiledbutnotexecuted,SoNginxhasnoideaavariable$blahisreferenced.Whenluacommandisexecutedinruntimebycommandcontent_by_lua,theluavariableisevaluatedasnil.Modulengx_luaanditscommandngx.saywillconvertLuanilintostring"nil"beforeitisprinted,sotheoutputwillbe:
curl'http://localhost:8080/test'
$blah=nil
Thisisindeedwhatwewant.
Weshouldhavenoticedalso,whencommandcontent_by_luaincludes$blahinitsparameter,itisneverevaluatedas"variableinterpolation"does(otherwiseNginxwillbecomplainingvariable$blahisnotdeclared).Thisisbecausecommandcontent_by_luadoesnotreallysupport"variableinterpolation".Aswehavesaidearlierin(01),Nginxcommanddoesnotnecessarilysupport"variableinterpolation"anditisentirelyuptothemoduleimplementation.
It'sactuallydifficulttoreturnan"invalid"non-value.Aswelearntin(07),variableswhicharedeclaredbutnotinitializedbysethasnon-value"invalid".However,assoonasthevariableisdevalued,the"gethandler"isexecutedandanemptystringiscomputedandcached,soeventuallyemptystringisreturned,notthe"invalid"non-value.Followingluacodecanprovethis:
location/foo{
content_by_lua'
ifngx.var.foo==nilthen
ngx.say("$fooisnil")
else
ngx.say("$foo=[",ngx.var.foo,"]")
end
';
}
location/bar{
set$foo32;
echo"foo=[$foo]";
}
Byrequestingtolocation/foowehave:
$curl'http://localhost:8080/foo'
$foo=[]
Aswecantell,whenLuareferencesuninitializedNginxvariable$foo,itobtainsemptystring.
Lastnottheleast,weshouldhavepointedout,althoughNginxvariablecanhaveonlystringsasvalidvalue.The3rdpartymodulengx_array_varcansupportarraylikeoperationsforNginxvariable.Hereisanexample:
location/test{
array_split","$arg_namesto=$array;
array_map"[$array_it]"$array;
array_join""$arrayto=$res;
echo$res;
}
Modulengx_array_varprovidescommandsarray_split,array_mapandarray_join.Thesemanticsisprettyclosetothebuiltinfunctionssplit,mapandjoininPerl(otherlanguagessupportsimilarfunctionalitiestoo).Nowlet'scheckwhathappenswhenlocation/testisrequested:
$curl'http://localhost:8080/test?names=Tom,Jim,Bob'
[Tom][Jim][Bob]
Clearlymodulengx_array_varmakeiteasiertohandleinputswithvariablelength,suchastheURLparametername,whichcomposesofmultiplecommadelimitednames.Stillwemustemphasize,modulengx_luaisamuchbetterchoicetoexecutethiskindofcomplicatedtasks,usuallyitismoreflexibleandmaintainable.
TillnowthetutorialcoverstheNginxvariable.Intheprocesswehavebeendiscussingmanybuiltinand3rdpartyNginxmodules,thesemoduleshelpusbetterunderstandfeaturesandinternalsofNginxvariablebycomposingvariousminiconstructs.Lateronthetutorialwillbecoveringmoredetailsofthosemodules.
Withtheseexamples,weshouldunderstandthatNginxvariableplaysakeyroleintheNginxminilanguage:variablesarethewaysandmeansNginxcommunicateinternally,theycontainalltheneededinformation(includingtherequestinformation)andtheyarethecornerstoneelementswhichbridgeeveryotherNginxmodules.Nginxvariablesareeverywhereinthecomingtutorials,understandthemisabsolutelynecessary.
Inthecomingtutorial"NginxDirectiveExecutionOrder",wewillbediscussingindetailtheNginxexecutionorderingandthephaseseveryrequesttraverses.It'sindispensabletounderstandthemsincefortheNginxminilanguage,theorderingofwritingcanbedramaticallydifferentfromtheorderingofexecutinginthetimeline.ItusuallyconfusesmanyNginxusers.
Nginxdirectiveexecutionorder(01)WhentherearemultipleNginxmodulecommandsinalocationdirective,theexecutionordercanbedifferentfromwhatyouexpect.BusyNginxuserswhoattempttoconfigureNginxby"trialanderror"maybeveryconfusedbythisbehavior.Thisseriesistouncoverthemysteriesandhelpyoubetterunderstandtheexecutionorderingbehindthescenes.
Westartwithaconfusedexample:
?location/test{
?set$a32;
?echo$a;
?
?set$a56;
?echo$a;
?}
Clearly,we'dexpecttooutput32,followedby56.Becausevariable$ahasbeenresetaftercommandecho"isexecuted".Really?therealityis:
$curl'http://localhost:8080/test
56
56
Wow,statementset$a56musthavehadbeenexecutedbeforethefirstecho$acommand,butwhy?IsitaNginxbug?
No,thisisnotanNginxbug.WhenNginxhandleseveryrequest,theexecutionfollowsafewpredefinedphases.
Therecanbealtogether11phaseswhenNginxhandlesarequest,let'sstartwiththreemostcommonones:rewrite,accessandcontent(Theotherphaseswillbeaddressedlater.)
UsuallyanNginxmoduleanditscommandsregistertheirexecutioninonlyoneofthosephases.Forexamplecommandsetrunsinphaserewrite,andcommandechorunsinphasecontent.Sincephaserewriteoccursbeforephasecontentforeveryrequestprocessing,itscommandsareexecutedearlieraswell.Therefore,commandsetalwaysgetsexecutedbeforecommandechowithinonelocationdirective,regardlessoftheirstatementorderingintheconfiguration.
Backtoourexample:
set$a32;
echo$a;
set$a56;
echo$a;
Theactualexecutionorderingis:
set$a32;
set$a56;
echo$a;
echo$a;
It'sclearnow,twocommandssetareexecutedinphaserewrite,twocommandsechoareexecutedafterwardsinphasecontent.Commandsindifferentphasescannotbeexecutedbackandforth.
Toprovethis,wecanenableNginx's"debuglog".
IfyouhavenotworkedwithNginx"debuglog"before,hereisabriefintroduction.The"debuglog"isdisabledbydefaultbecauseperformanceisdegradedwhenitisenabled.Toenable"debuglog"youmustreconfigureandrecompileNginx,andsetthe--with-debugoptionforthepackage's./configurescript.WhenbuildingunderLinuxorMacOSXfromsource:
tarxvfnginx-1.0.10.tar.gz
cdnginx-1.0.10/
./configure--with-debug
make
sudomakeinstall
Incasethepackagengx_openrestyisused.Theoption--with-debugcanbeusedwithits./configurescriptaswell.
AfterwerebuildtheNginxdebugbinarywith--with-debugoption,westillneedtoexplicitlyusethedebugloglevel(it'sthelowestlevel)forcommanderror_log,inNginxconfiguration:
error_loglogs/error.logdebug;
debug,thesecondparameterofcommanderror_logiscrucial.Itsfirstparameteriserrorlog'sfilepath,logs/error.log.Certainlywecanuseanotherfilepathbutdorememberthelocationbecauseweneedtocheckitscontentrightaway.
Nowlet'srestartNginx(Attention,it'snotenoughtoreloadNginx.Itneedstobekilledandrestartedbecausewe'veupdatedtheNginxbinary).Thenwecansendtherequestagain:
$curl'http://localhost:8080/test'
56
56
It'stimetocheckNginx'serrorlog,whichisbecomingalotmoreverbose(morethan700linesfortherequestinmysetup).Solet'sapplythegrepcommandtofilterwhatwewouldbeinterested:
grep-E'http(outputfilter|script(set|value))'logs/error.log
It'sapproximatelylikebelow(forclearness,I'veeditedthegrepoutputandremoveitstimestampetc):
[debug]5363#0:*1httpscriptvalue:"32"
[debug]5363#0:*1httpscriptset$a
[debug]5363#0:*1httpscriptvalue:"56"
[debug]5363#0:*1httpscriptset$a
[debug]5363#0:*1httpoutputfilter"/test?"
[debug]5363#0:*1httpoutputfilter"/test?"
[debug]5363#0:*1httpoutputfilter"/test?"
Itbarelymakesanysenses,doesit?Soletmeinterpret.Commandsetdumpstwolinesofdebuginfowhichstartwithhttpscript,thefirstlinetellsthevaluewhichcommandsethaspossessed,andthesecondlinebeingthevariablenameitwillbegivento,sofortheleadingfilteredlog:
[debug]5363#0:*1httpscriptvalue:"32"
[debug]5363#0:*1httpscriptset$a
Thesetwolinesaregeneratedbythisstatement:
set$a32;
Andforthefollowingfilteredlog:
[debug]5363#0:*1httpscriptvalue:"56"
[debug]5363#0:*1httpscriptset$a
Theyaregeneratedbythisstatement:
set$a56;
Besides,wheneverNginxoutputsitsresponse,its"outputfilter"willbeexecuted,ourfavoritecommandechoisnoexception.AssoonasNginx's"outputfilter"isexecuted,itgeneratesdebugloglikebelow:
[debug]5363#0:*1httpoutputfilter"/test?"
Ofcoursethedebuglogmightnothave"/test?",sincethispartcorrespondstotheactualrequestURI.Byputtingeverythingtogether,wecanfinallyconcludethosetwocommandssetareindeedexecutedbeforetheothertwocommandsecho.
Consideratereadersmusthavenoticedthattherearethreelinesofhttpoutputfilterdebuglogbutwewerehavingonlytwooutputcommandsecho.Infact,onlythefirsttwodebuglogsaregeneratedbythetwoechostatements.Thelastdebuglogisaddedbymodulengx_echobecauseitneedstoflagtheendofoutput.TheflagoperationitselfcausesNginx's"outputfilter"tobeexecutedagain.Manymodulesincludingngx_proxyhassimilarbehavior,whentheyneedtogiveoutputdata.
Allright,therearenosurpriseswiththoseduplicated56outputs.Wearenotgivenachancetoexecuteechoinfrontofthesecondsetcommand.Luckily,wecanstillachievethiswithafewtechniques:
location/test{
set$a32;
set$saved_a$a;
set$a56;
echo$saved_a;
echo$a;
}
Nowwehavewhatwehavewanted:
$curl'http://localhost:8080/test'
32
56
Withthehelpofanotheruservariable$saved_a,thevalueof$aissavedbeforeitisoverwritten.Becareful,theexecutionorderofmultiplesetcommandsareensuredtobeliketheirorderofwritingbymodule.Similarly,modulengx_echoensuresmultipleechocommandsgetexecutedinthesameorderoftheirwriting.
IfwerecallexamplesinNginxVariables,thistechniquehasbeenappliedextensively.ItbypassestheexecutionorderingdifficultiesintroducedbyNginxphasedprocessing.
Youmightneedtoask:"howwouldIknowthephaseaNginxcommandbelongsto?"Indeed,theanswerisRTFD.(SurelyadvanceddeveloperscanexaminetheCsourcecodedirectly).Manymodulemarksexplicitlyitsapplicablephaseinthemodule'sdocumentation,suchascommandechowritesbelowinitsdocumentation:
phase:content
Itsaysthecommandisexecutedinphasecontent.Ifyouencountersamodulewhichmissestheapplicablephaseinthedocument,youcanwritetoitsauthorsrightawayandaskforit.However,weshallbereminded,noteverycommandhasanapplicablephase.ExamplesarecommandgeointroducedinNginxVariables(01)andcommandmapintroducedinNginxVariables(04).Thesecommands,whohavenoexplicitapplicablephase,aredeclarativeandunrelatedtotheconceptionofexecutionordering.IgorSysoev,theauthorofNginx,hasmadethestatementsafewtimespublicly,thatNginxminilanguageinitsconfigurationis"declarative"not"procedural".
Nginxdirectiveexecutionorder(02)We'vejustlearnt,allsetcommandswithinlocationareexecutedinrewritephase.Infact,almostallcommandsimplementedbymodulerewriteareexecutedinrewritephaseunderthespecificcontext.CommadrewriteintroducedinNginxVariables(02)isoneofthem.However,weshallpointoutthatwhenthesecommandsarefoundinserverdirective,theywillbeexecutedinanearlierphasewe'venotaddressed:theserverrewritephase.
Commandset_unescape_uri,introducedinNginxVariables(02)isalsoexecutedinrewritephase.Actually,commandsimplementedbymodulengx_set_misccanmixwithcommandsimplementedbymodulengx_rewriteandtheexecutionorderingisensured.Let'scheckanexample:
location/test{
set$a"hello%20world";
set_unescape_uri$b$a;
set$c"$b!";
echo$c;
}
Bysendingarequestaccordinglywehave:
$curl'http://localhost:8080/test'
helloworld!
Apparently,theset_unescape_uricommandanditsneighboringsetcommandsareallexecutedintheorderoftheirwriting.
Tofurtherdemonstrateourassertion,wecheckagainNginx"debuglog"(incaseit'sunclearforyouhowtocheck"debuglog",pleasereferencestepsfoundin(01)).
grep-E'httpscript(value|copy|set)'logs/error.log
Thedebuglogsarefilteredas:
[debug]11167#0:*1httpscriptvalue:"hello%20world"
[debug]11167#0:*1httpscriptset$a
[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"
[debug]11167#0:*1httpscriptset$b
[debug]11167#0:*1httpscriptcopy:"!"
[debug]11167#0:*1httpscriptset$c
Theleadingtwolines:
[debug]11167#0:*1httpscriptvalue:"hello%20world"
[debug]11167#0:*1httpscriptset$a
Theycorrespondtothecommand
set$a"hello%20world";
Thefollowingtwolines:
[debug]11167#0:*1httpscriptvalue(postfilter):"helloworld"
[debug]11167#0:*1httpscriptset$b
Theyaregeneratedbycommand
set_unescape_uri$b$a;
Thereareminordifferencesinthefirstline,ifwecomparetothelogsgeneratedbycommandset:the"(postfilter)"addition.Intheendoftheline,URLdecodinghassuccessfullyexecutedaswewish."hello%20world"isdecodedas"helloworld".
Thelasttwolinesofdebuglog:
[debug]11167#0:*1httpscriptcopy:"!"
[debug]11167#0:*1httpscriptset$c
Theyaregeneratedbythelastsetcommand
set$c"$b!";
Asyoumighthavenoticed,since"variableinterpolation"isevaluatedwhenvariable$cisdeclaredandinitialized,thedebuglogstartswithhttpscriptcopy.Intheendofthelogitisthestringconstant"!"tobeconcatenated.
Withtheloginformation,it'sfairlyeasytotellthecommandexecutionordering:
set$a"hello%20world";
set_unescape_uri$b$a;
set$c"$b!";
Itisaperfectmatchtothestatementsordering.
Justlikethecommandsimplementedinmodulengx_set_misc,commandset_by_luaimplementedin3rdpartymodulengx_lua,canmixwithcommandsofmodulengx_rewriteaswell.AsintroducedinNginxVariables(07),commandset_by_luasupportscomputationwithgivenLuacode,andassignsthecomputedresulttoaNginxvariable.Ascommandsetdoes,commandset_by_luadeclaresNginxvariablebeforeinitializationifthevariabledoesnotexist.
Let'scheckamixedexamplewhichcomprisescommandset_by_luaandset:
location/test{
set$a32;
set$b56;
set_by_lua$c"returnngx.var.a+ngx.var.b";
set$equation"$a+$b=$c";
echo$equation;
}
Variable$aand$bareinitializedwithnumericalvalue32and56respectively,thencommandset_by_luaisusedtogetherwithgivenLuacodetocomputethesumof$aand$b.Variable$cisinitializedwiththecomputedvalue.Finally,variables$a,$band$careconcatenatedby"variableinterpolation"andassignstheresulttovariable$equation,whichisprintedbycommandecho.
Weshallpayattentiontoafewpointsintheexample:FirstlyNginxvariable$VARIABLEisreferencedasngx.var.VARIABLEinLuacode.Secondly,sinceNginxvariablesarestrings,thevalueofvariablengx.var.aandngx.var.bareactuallystrings"32"and"56",howevertheyareautomaticallyconvertedtonumericalvaluesbyLuaintheadditionoperation.ThirdlyLuacodereturnstoNginxvariable$cthecomputedsumvaluebystatementreturn.FinallywhenLuacodereturns,itactuallyconvertsthenumericalvaluebacktostring.(becausestringistheonlyvalidvalueforNginxvariable)
Theactualoutputmeetsourexpectation:
$curl'http://localhost:8080/test'
32+56=88
Thisinfactassertsthatcommandset_by_luacanmixwithcommandsimplementedbymodulengx_rewrite,suchasset.
Manyother3rdpartymodulessupportthemixwithmodulengx_rewriteaswell.Theexamplesincludemodulengx_array_var,discussedinNginxVariables(08)andmodulengx_encrypted_session,whichencryptssessions.Thelatterwillbestudiedindetailshortly.
Sincebuiltinmodulengx_rewriteisvirtuallyindispensable,it'sagreatadvantageforthe3rdpartymodulehasthecaliberofbeingmixedwith.Truthis,allofthose3rdpartymoduleshaveadoptedaspecialtechnique,whichallowsthe"injection"oftheirexecutionintocommandsofmodulerewrite(withthehelpofa3rdpartymodulengx_devel_kitdevelopedbyMarcusClyne).Fortherestregular3rdpartymodules,whichalsoregistertheirexecutioninphaserewrite,theircommandsareexecutedseparatelyfrommodulengx_rewriteinruntime.Infact,it'shardlyaccuratetotellthecommandsexecutionorderinginbetweendifferentmodules(strictlyspeakingtheyareusuallyexecutedintheorderofloading,butexceptiondoesexist).Forexamplebothmodules,AandBregistertheircommandstobeexecutedinphaserewrite,thenitiseitherthecaseinwhichcommandsofAareexecutedfollowedbyBortheothercompletewayaround.Unlessitisexplicitlydocumented,wecannotrelyontheuncertainorderinginourconfigurations.
Nginxdirectiveexecutionorder(03)Asdiscussedearlier,unlessspecialtechniquesareutilizedasmodulengx_set_miscdoes,amodulecannotmixitscommandswithngx_rewrite,andexpectsthecorrectexecutionorder.Evenifthecommandsareregisteredintherewritephaseaswell.Wecandemonstratewithsomeexamples.
3rdpartymodulengx_headers_moreprovidesafewcommands,whichdealwiththecurrentrequestheaderandresponseheader.Oneofthemismore_set_input_header.Thecommandcanmodifyagivenrequestheaderinrewritephase(oraddthespecificheaderifit'snotavailableincurrentrequest).Asdescribedinitsdocumentation,thecommandalwaysexecutesintheendofrewritephase:
phase:rewritetail
Beingtersethough,rewritetailmeanstheendofphaserewrite.
Sinceitexecutesintheendofphaserewrite,theimplicationisitsexecutionisalwaysafterthecommandsimplementedinmodulengx_rewrite.Evenifitiswrittenattheverybeginning:
?location/test{
?set$valuedog;
?more_set_input_headers"X-Species:$value";
?set$valuecat;
?
?echo"X-Species:$http_x_species";
?}
AsbrieflyintroducedinNginxVariables(02),Builtinvariable$http_XXXhastheheaderXXXforthecurrentrequest.Wemustbecarefulthough,variable<$http_XXX>matchestothenormalizedrequestheader,i.e.itlowercasescapitallettersandturnsminus-intounderscore_fortherequestheadernames.Thereforevariable$http_x_speciescansuccessfullycatchestherequestheaderX-Species,whichisdeclaredbycommandmore_set_input_header.
Becauseofthestatementordering,wemighthavemistakenlyconcludedheaderX-Specieshasthevaluedogwhen/testisrequested.Buttheactualresultisdifferent:
$curl'http://localhost:8080/test'
X-Species:cat
Clearly,statementset$valuecatisexecutedearlierthanmore_set_input_headers,althoughitiswrittenafterwards.
Thisexampletellsusthatcommandsofdifferentmodulesareexecutedindependentlyfromeachother,eveniftheyareallregisteredinthesameprocessingphase.(unlessitisimplementedasmodulengx_set_misc,whosecommandsarespecificallytunedwithmodulengx_rewrite).Inotherwords,everyprocessingphaseisfurtherdividedintosub-phasesbyNginxmodules.
Similartomore_set_input_headers,commandrewrite_by_luaprovidedby3rdpartymodulengx_luaexecuteintheendofrewritephaseaswell.Wecanverifythis:
?location/test{
?set$a1;
?rewrite_by_lua"ngx.var.a=ngx.var.a+1";
?set$a56;
?
?echo$a;
?}
ByusingLuacodespecifiedbycommandrewrite_by_luaNginxvariable$aisincrementedby1.Wemighthaveexpectedtheresultbe56ifwearelookingatthewritingsequence.Theactualresultis57becausecommandisalwaysexecutedafterallthesetstatements.
$curl'http://localhost:8080/test'
57
Admittedlycommandrewrite_by_luahasdifferentbehaviorthancommandset_by_lua,whichisdiscussedin(02).
Outofsheercuriosity,weshallaskimmediatelythatwhatwouldbeexecutionorderinginbetweenmore_set_input_headersandrewrite_by_lua,sincetheybothrideonrewritetail?Theansweris:undefined.Wemustavoidaconfigurationwhichreliesontheirexecutionorders.
Nginxphaserewriteisaratherearlyprocessingphase.Usuallycommandsregisteredinthisphaseexecutevariousrewritetasksontherequest(forexamplerewritetheURLortheURLparameters),thecommandsmightalsodeclareandinitializeNginxvariableswhichareneededinthesubsequenthandling.Certainly,onecannotforbidotherstocomplicatethemselvesbycheckingtherequestbody,orvisitadatabaseetc.Afterall,commandlikerewrite_by_luaoffersthecalibertostuffinanypotentiallymindtwistedLuacode.
Afterphaserewrite,Nginxhasanotherphasecalledaccess.Thecommandsprovidedby3rdpartymodulengx_auth_request,whichisdiscussedinNginxVariables(05),executeinphaseaccess.CommandsregisteredinaccessphasemostlycarryoutACLfunctionalities,suchasguardinguserclearance,checkinguserorigins,examiningsourceIPvalidityetc.
Forexamplecommandallowanddenyprovidedbybuiltinmodulengx_accesscancontrolwhichIPaddresseshavetheprivilegestovisit,orwhichIPaddressesarerejected:
location/hello{
allow127.0.0.1;
denyall;
echo"helloworld";
}
Location/helloallowsvisitfromlocalhost(IPaddress127.0.0.1)andrejectrequestsfromallotherIPaddresses(returnshttperror403)Therulesdefinedbyngx_accesscommandsareassertedinthewritingsequence.Onceoneruleismatched,theassertionstopsandalltherestallowordenycommandsareignored.Ifno
ruleismatched,handlingcontinuesinthefollowingstatements.Ifthematchedruleisdeny,handingisabortedanderror403isreturnedimmediately.Inourexample,requestissuedfromlocalhostmatchestotheruleallow127.0.0.1andhandingcontinuestotheotherstatements,howeverrequestissuedfromeveryotherIPaddresseswillmatchruledenyallhandlingisthereforeabortedanderror403isreturned.
Wecangiveitatest,bysendingrequestfromlocalhost:
$curl'http://localhost:8080/hello'
helloworld
Ifrequestissentfromanothermachine(supposeNginxrunsonIP192.168.1.101)wehave:
$curl'http://192.168.1.101:8080/hello'
<html>
<head><title>403Forbidden</title></head>
<bodybgcolor="white">
<center><h1>403Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
Bytheway,modulengx_accesssupportsthe"CIDRnotation"todesignateasub-network.Forexample169.200.179.4/24representsthesub-networkwhichhastheroutingprefix169.200.179.0(orsubnetmask255.255.255.0)
Becausecommandsofmodulengx_accessexecuteinaccessphase,andphaseaccessisbehindrewritephase.Soforthosecommandswehavebeendiscussing,regardlessofthewritingordertheyalwaysexecuteinrewritephase,whichisearlierthanallowordeny.Keepthisinmind,weshalltryourbesttokeepthewritingandexecutionorderconsistent.
Nginxdirectiveexecutionorder(04)Modulengx_luaimplementsanothercommandaccess_by_lua.Thecommandallowsluacodetobeexecutedintheendofaccessphase,whichmeansitalwaysexecutesafterallowanddenyeventheybelongtothesamephase.Inmanycases,weexaminetherequest'ssourceIPaddresswithngx_access,andusecommandaccess_by_luatoexecutemorecomplicatedverificationswithLua.Forexamplebyqueryingadatabaseorotherbackendservices,thecurrentuser'sidentityandprivilegesareexamined.
Wecancheckasimpleexample,whichusescommandaccess_by_luatoimplementtheIPfilteringfunctionalityofmodulengx_access
location/hello{
access_by_lua'
ifngx.var.remote_addr=="127.0.0.1"then
return
end
ngx.exit(403)
';
echo"helloworld";
}
Nginx'sbuiltinvariable$remote_addrisreferencedinLuatogettheclient'sIPaddress.ThenLuastatementifisusedtodetermineiftheaddressequals127.0.0.1.Luareturnsifitequals,Nginxthuscontinuesthesubsequenthandling(includingthecontentphasewherecommandechoappliesto).Ifitisnotthelocalhostaddress,currenthandlingisabortedbyusingngx_luamodule'sLuafunctionngx.exitClientgetsahttperror403.
Theexampleisequivalenttotheotherexampleusingngx_accessmoduleintermsoffunctionality,whichwasdiscussedin(03):
location/hello{
allow127.0.0.1;
denyall;
echo"helloworld";
}
Howeverweshallpointout,performancewisethetwostillhavedifferences.Modulengx_accessperformsbetterbecauseitisspecificallyimplementedasaNginxmoduleinC.
Wecanmeasuretheperformancedifferencesofthetwo.Afterall,performanceiswhatweareafterbyusingNginx.Ontheotherhand,it'sabsolutelynecessarytobeequippedwithmeasuringtechniques,becauseonlyactualdatadistinguishesamateursandprofessionals.Infact,bothngx_luaandngx_accessperformprettygoodforIPfiltering.Tominimizemeasuringerrorswecouldmeasuredirectlytheelapsedtimeofaccessphase.Traditionally,thismeanshackingNginxsourcecodewithtimingcodeandstatisticalcode,orrecompileNginxbinarysothatitcanbemonitoredbyspecificprofilingtoolslikeGNUgprof.
Wearelucky,becausecurrentreleasesofSolaris,MacOSXorFreeBSDofferasystemutilitydtrace,whichallowsmicromonitoringofuserprocessintermsofperformance(andfunctionalityaswell).Thetoolsparesusfromhackingsourcecodeorrecompilationwithprofiling.Let'sdemonstratethemeasuringscenarioontheMacBookAirbecausedtraceisavailablesinceMacOSX10.5
First,opentheTerminalapplicationofMacOSX,changetoyourpreferablepathandcreateafilenamedasnginx-access-time.d,editthefilewithfollowingcontent:
#!/usr/bin/envdtrace-s
pid$1::ngx_http_handler:entry
{
elapsed=0;
}
pid$1::ngx_http_core_access_phase:entry
{
begin=timestamp;
}
pid$1::ngx_http_core_access_phase:return
/begin>0/
{
elapsed+=timestamp-begin;
begin=0;
}
pid$1::ngx_http_finalize_request:return
/elapsed>0/
{
@elapsed=avg(elapsed);
elapsed=0;
}
Savethefileandmakeitexecutable.
$chmod+x./nginx-access-time.d
The.dfileactuallycontainscodewritteninDlanguageofferedbyutilitydtrace(attention,theDlanguageisnottheotherDlanguage,whichisadvocatedbyWalterBrightforabetterC++).SofarwecannotreallyexplainindetailthecodebecauseitrequiresathoroughunderstandingofNginxinternals.Anywayweshallbeclearof
thecode'spurpose:measurerequestsbeinghandledbyspecificNginxworkerprocessandcalculatetheaveragetimeelapsedinaccessphase.
NowwecangettheDscriptrunning.Thescripttakesacommandlineparameter,whichistheprocessid(pid)ofNginxworker.SinceNginxsupportsmultipleworkerprocessesandtherequestscanberandomlyhandledbyanyoneofthem,we'dliketoconfigureNginxinitsconfigurationnginx.confsothatonlyoneworkerisrequested.
worker_processes1;
AfterNginxbinaryisrestarted,theworkerprocessidcanbeobtainedbycommandps.
$psax|grepnginx|grepworker|grep-vgrep
Typicallywehave:
10975??S0:34.28nginx:workerprocess
10975ismyNginxworkerpid.Incaseyouhavemultiplelines,youmusthavestartedmultipleNginxserverinstancesorthecurrentNginxserverhasstartedmultipleworkerprocesses.
Thenasroot,scriptnginx-access-time.disexecutedwiththeworkerpid
$sudo./nginx-access-time.d10975
WeshallhaveoneoutputmessageifeverythinggoesOK.
dtrace:script'./nginx-access-time.d'matched4probes
ThemessagesaysourDscripthassuccessfullydeployed4probesonthetargetprocess.Thenthescriptisreadytotraceprocess10975constantly.
Let'sopenanotherTerminal,andsendmultiplerequestswithcurltoourmonitoredprocess
$curl'http://localhost:8080/hello'
helloworld
$curl'http://localhost:8080/hello'
helloworld
BacktoourTerminalwhereDscriptisrunning,presskeysCtrl-Ctointerruptit.Whenthescriptbailsoutitprintsonconsolethestatisticalresult.Forexamplemyconsolehasfollowingresult:
$sudo./nginx-access-time.d10975
dtrace:script'./nginx-access-time.d'matched4probes
^C
19219
Thefinal19219istheaveragetimeelapsedinaccessphaseinnanoseconds(1second=1000x1000x1000nanoseconds)
Donewiththesteps.Wecanrunthenginx-access-time.dscripttocalculateaverageelapsedtimeinphaseaccessforthreedifferentNginxsetupsrespectively.TheyareIPfilteringwithmodulengx_access,IPfilteringwithcommandaccess_by_lua,andfinallynofilteringforaccessphase.Thelastresulthelpseliminatethesideeffectcausedbyprobesorother"systematicerrors".Besides,wecanusetrafficloadertoolssuchasabtosendshalfamillionrequeststominimize"randomerrors",asbelow:
$ab-k-c1-n100000'http://127.0.0.1:8080/hello'
ThereforethestatisticalresultofDscriptisascloseaspossibletothe"actual"time.
IntheMacOSX,atypicalrunhasfollowingresults:
ngx_access18146
access_by_lua35011
nofiltering15887
Weminusthelastvaluefromtheformertwo:
ngx_access2259
access_by_lua19124
Well,modulengx_accessoutperformscommandaccess_by_luabyamagnitude,aswemighthaveexpected.Stilltheabsolutedifferenceistiny.FortheIntelCore2Due1.86GHzCPUofmine,thereisonlyafewmicroseconds.
Infacttheaccess_by_luaexamplecanbefurtheroptimizedusingbuiltinvariable$binary_remote_addr.ThisvariablehastheIPaddressinbinaryformwhereasvariable$remote_addrhastheaddressinalongerstringformat.ShorteraddresscanbecomparedquickerwhenLuaexecutesitsstringoperations.
Becareful,if"debuglog"isenabledasintroducedin(01)thecomputedelapsedtimewillincreasedramatically,because"debuglog"hasahugeoverhead.
Nginxdirectiveexecutionorder(05)contentisbyallmeansthemostsignificantphaseinNginx'srequesthandling,becausecommandsrunninginthephasehavetheresponsibilitytogenerate"content"andoutputHTTPresponse.Becauseofitsimportance,Nginxhasarichsetofcommandsrunninginit.Thecommandsincludeecho,echo_exec,proxy_pass,echo_location,content_by_lua,whichwerediscussedinNginxVariables(02),NginxVariables(03),NginxVariables(05)andNginxVariables(07)respectively.
contentisaphasewhichrunslaterthanrewriteandaccess.Thereforeitscommandsalwaysexecuteintheendwhentheyareusedtogetherwithcommandsofrewriteandaccess.
location/test{
#rewritephase
set$age1;
rewrite_by_lua"ngx.var.age=ngx.var.age+1";
#accessphase
deny10.32.168.49;
access_by_lua"ngx.var.age=ngx.var.age*3";
#contentphase
echo"age=$age";
}
Thisisaperfectexample,inwhichcommandsareexecutedinanexactsequenceastheyarewritten.Thetestingresultmatchestoourexpectationstoo.
$curl'http://localhost:8080/test'
age=6
Infact,thecommands'writingordercanbecompletelyshuffledanditwon'thaveanyimpacttotheirexecutionsequence.Commandset,whichisimplementedbymodulengx_rewrite,executesinrewritephase.Commandrewrite_by_luafrommodulengx_luaexecutesintheendofrewritephase.Commanddenyfrommodulengx_accessexecutesinaccessphase.Commandaccess_by_luafrommodulengx_luaexecutesintheendofaccessphase.Finally,ourfavoritecommandecho,implementedbymodulengx_echo,executesincontentphase.
TheexamplealsodemonstratesthecollaboratinginbetweencommandsrunningoneachdifferentNginxphase.Intheprocess,Nginxvariableisthedatacarrierinterconnectingcommandsandmodules.Theexecutionorderofthesecommandsislargelydecidedbythephaseeachappliesto.
Asmatteroffact,multiplecommandsfromdifferentmodulescouldcoexistinphaserewriteandaccess.Astheexampleshows,commandsetandcommandrewrite_by_luabothbelongtophaserewrite.Commanddenyandcommandaccess_by_luabothbelongtophaseaccess.Howeveritisnotthesamestoryforphasecontent.
Mostmodules,whentheyimplementcommandsforphasecontent,theyareactuallyinserting"contenthandler"forthecurrentlocationdirective,howevertherecanbeoneandonlyone"contenthandler"foralocation.Soonlyonemodulecouldbeattherestwhenmultiplemodulesarecontendingtherole.Considerfollowingproblematicexample:
?location/test{
?echohello;
?content_by_lua'ngx.say("world")';
?}
Commandechofrommodulengx_echoandcommandcontent_by_luafrommodulengx_luabothexecuteinphasecontent.Butonlyoneofthemcouldsuccessfullybecome"contenthandler":
$curl'http://localhost:8080/test'
world
Ourtestindicates,thatthewinneriscontent_by_luaalthoughitiswrittenafterwards,andcommandechoneverreallyhasachancetorun.Wecannotbeassuredwhichmodulewinsinthecircumstance.Forexample,modulengx_echowinsandtheoutputbecomeshelloifweswapthecontent_by_luaandechostatements.Soweshallavoidtousemultiplecommandsforphasecontent,ifthecommandsareimplementedbydifferentmodules.
Theexamplecanbemodifiedbyreplacingcommandcontent_by_luawithcommandechoandwewillgetwhatweneed:
location/test{
echohello;
echoworld;
}
Againtestproves:
$curl'http://localhost:8080/test'
hello
world
Wecanusemultipleechocommands,thereisnoproblemwiththisbecausetheyallbelongtomodulengx_echo.Modulengx_echoregulatestheexecutionorderingofthem.Becarefulthough,noteverymodulesupportsthecommandsbeingexecutedmultipletimeswithinonelocation.Commandcontent_by_luaforaninstance,canbeusedonlyonce,sofollowingexampleisincorrect:
?location/test{
?content_by_lua'ngx.say("hello")';
?content_by_lua'ngx.say("world")';
?}
Nginxdumpserrorfortheconfiguration:
[emerg]"content_by_lua"directiveisduplicate...
Thecorrectwayofdoingitis:
location/test{
content_by_lua'ngx.say("hello")ngx.say("world")';
}
Insteadofusingtwicethecontent_by_luacommandinlocation,theapproachistocallfunctionngx.saytwiceintheLuacode,whichisexecutedbycommandcontent_by_lua
Similarly,commandproxy_passfrommodulengx_proxycannotcoexistwithcommandechowithinonelocationbecausetheybothexecuteincontentphase.ManyNginxnewbiesmakefollowingmistake:
?location/test{
?echo"before...";
?proxy_passhttp://127.0.0.1:8080/foo;
?echo"after...";
?}
?
?location/foo{
?echo"contentstobeproxied";
?}
Theexampletriestooutputstrings"before..."and"after..."withcommandechobeforeandaftermodulengx_proxyreturnsitscontent.Howeveronlyonemodulecouldexecuteincontent.Thetestindicatesmodulengx_proxywinsandcommandechofrommodulengx_echoneverruns
$curl'http://localhost:8080/test'
contentstobeproxied
Toimplementwhattheexamplehadwantedto,weshallusetwoothercommandsprovidedbymodulengx_echo,echo_before_bodyandecho_after_body:
location/test{
echo_before_body"before...";
proxy_passhttp://127.0.0.1:8080/foo;
echo_after_body"after...";
}
location/foo{
echo"contentstobeproxied";
}
Testtellswemakeit:
$curl'http://localhost:8080/test'
before...
contentstobeproxied
after...
Thereasoncommandsecho_before_bodyandecho_after_bodycouldcoexistwithothermodulesincontentphase,istheyarenot"contenthandler"but"outputfilter"ofNginx.Backin(01)whenweexaminethe"debuglog"generatedbycommandecho,we'velearntNginxcallsits"outputfilter"wheneverNginxoutputsdata.Sothatmodulengx_echotakestheadvantageofittomodifycontentgeneratedbymodulengx_proxy(byaddingsurroundingcontent).Weshallpointoutthough,"outputfilter"isnotoneofthose11phasesmentionedin(01)(manyphasescouldtrigger"outputfilter"whentheyoutputdata).Stillit'sperfectlyallrighttodocumentcommandsecho_before_bodyandecho_after_bodyasfollowing:
phase:outputfilter
Itmeansthecommandexecutesin"outputfilter".
Nginxdirectiveexecutionorder(06)We'velearntin(05)thatwhenacommandexecutesincontentphaseforaspecificlocation,itusuallymeansitsNginxmoduleregistersa"contenthandler"forthelocation.However,whathappensifnomoduleregistersitscommandas"contenthandler"forphasecontent?Whowillbetakingthegloryofgeneratecontentandoutputresponses?Theansweristhestaticresourcemodule,whichmapstherequestURItothefilesystem.Staticresourcemoduleonlycomesintoplaywhenthereisnone"contenthandler",otherwiseithandsoffthedutyto"contenthandler".
TypicallyNginxhasthreestaticresourcemodulesforthecontentphase(unlessoneormoreofthosemodulesaredisabledexplicitly,orsomeotherconflictingmodulesareenabledwhenNginxisbuilt)Thethreemodules,intheorderoftheirexecutionorder,arengx_indexmodule,ngx_autoindexmoduleandngx_staticmodule.Let'sdiscussthemonebyone.
Modulengx_indexandngx_autoindexonlyapplytothoserequestURI,whichendswith/.FortheotherrequestURIwhichdoesnotendwith/,bothmodulesignorethemandletthefollowingcontentphasemodulehandle.Modulengx_statichowever,hasanexactoppositestrategy.ItignorestherequestURIwhichendswith/andhandlestherest.
Modulengx_indexmainlylooksforaspecifichomepagefile,suchasindex.htmlorindex.htminthefilesystem.Forexample:
location/{
root/var/www/;
indexindex.htmindex.html;
}
Whenaddress/isrequested,Nginxlooksforfileindex.htmandindex.html(inthisorder)inapathinthefilesystem.Thepathisspecifiedbycommandroot.Iffileindex.htmexists,Nginxjumpsinternallytolocationindex.htm;ifitdoesnotexistandfileindex.htmlexists,Nginxjumpsinternallytolocationindex.html.Iffileindex.htmldoesnotexisteither,andhandlingistransferredtotheothermodulewhichexecutesitcommandsinphasecontent.
WehavelearntinNginxVariables(02),commandsecho_execandrewritecantrigger"internalredirects"aswell.ThejumpmodifiestherequestURI,andlooksforthecorrespondinglocationdirectiveforsubsequenthandling.Intheprocess,phasesrewrite,accessandcontentarereiteratedforthelocation.The"internalredirect"isdifferentfromthe"externalredirect"definedbyHTTPresponsecode302and301,clientbrowserwon'tupdateitsURIaddresses.Thereforeassoonasinternaljumpoccurswhenmodulengx_indexfindsthefilesspecifiedbycommandindex,theneteffectislikeclientwouldhavebeenrequestingthefile'sURIattheverybeginning.
Wecancheckfollowingexampletowitnessthe"internalredirect"triggeredbymodulengx_index,whenitfindstheneededfile.
location/{
root/var/www/;
indexindex.html;
}
location/index.html{
set$a32;
echo"a=$a";
}
Weneedtocreateanemptyfileindex.htmlunderthepath/var/www/,andmakesurethefileisreadablefortheNginxworkerprocess.Thenwecouldsendrequestto/:
$curl'http://localhost:8080/'
a=32
Whathappened?Whytheoutputisnotthecontentoffileindex.html(whichshallbeempty)?FirstlyNginxusesdirectivelocation/tohandleoriginalGET/request,thenmodulengx_indexexecutesincontentphase,anditfindsfileindex.htmlunderpath/var/www/.Atthismoment,ittriggersan"internalredirect"tolocation/index.html.
Sofarsogood.Butherecomesthesurprises!WhenNginxlooksforlocationdirectivewhichmatchesto/index.html,location/index.htmlhasahigherprioritythanlocation/.ThisisbecauseNginxuses"longestmatchedsubstring"semanticstomatchlocationdirectivestorequestURI'sprefix.Whendirectiveischosen,phasesrewrite,accessandcontentarereiterated,andeventuallyitoutputsa=32.
Whatifweremovefile/var/www/index.htmlintheexample,andrequestto/again?Theansweriserror403Forbidden.Why?Whenmodulengx_indexcannotfindthefilespecifiedbycommandindex(index.htmlinhere),ittransfersthehandlingtothefollowingmodulewhichexecutesincontent.Butnoneofthosefollowingmodulescanfulfilltherequest,Nginxbailsoutanddumpsuserror.MeanwhileitlogstheerrorinNginxerrorlog:
[error]28789#0:*1directoryindexof"/var/www/"isforbidden
Themeaningofdirectoryindexistogenerate"indexes".Usuallythisimpliestogenerateawebpage,whichlistseveryfileandsubdirectoriesunderpath/var/www/.Ifweusemodulengx_autoindexrightafterngx_index,itcangeneratesuchapagejustlikewhatweneed.Nowlet'smodifytheexamplealittlebit:
location/{
root/var/www/;
indexindex.html;
autoindexon;
}
When/isrequestedagainmeanwhilefile/var/www/index.htmliskeptmissing.Anicehtmlpageisgenerated:
$curl'http://localhost:8080/'
<html>
<head><title>Indexof/</title></head>
<bodybgcolor="white">
<h1>Indexof/</h1><hr><pre><ahref="../">../</a>
<ahref="cgi-bin/">cgi-bin/</a>08-Mar-201019:36-
<ahref="error/">error/</a>08-Mar-201019:36-
<ahref="htdocs/">htdocs/</a>05-Apr-201003:55-
<ahref="icons/">icons/</a>08-Mar-201019:36-
</pre><hr></body>
</html>
Thepageshowsthereareafewsubdirectoriesundermy/var/www/.Theyarecgi-bin/,error/,htdocs/andicons/.Theoutputmightbedifferentifyouhavetriedbyyourself.
Again,iffile/var/www/index.hmtldoesexist,modulengx_indexwilltrigger"internalredirect",andmodulengx_autoindexwillnothaveachancetoexecute,youmaytestityourselftoo.
The"goalkeeper"moduleexecutedinphasecontentisngx_static.whichisalsousedintensively.Themoduleservesthestaticfiles,includingthestaticresourcesofawebsite,suchasstatic.htmlfiles,static.cssfiles,static.jsfilesandstaticimagefilesetc.Althoughngx_indexcouldtriggeran"internalredirect"tothespecifiedhomepage,buttheactualoutputtask(takesthefilecontentasresponse,andmarksthecorrespondingresponseheaders)iscarriedoutbymodulengx_static.
Nginxdirectiveexecutionorder(07)Let'scheckanexampleinwhichmodulengx_staticservesdiskfiles,withfollowingconfigurationsnippet:
location/{
root/var/www/;
}
Meanwhiletwofilesarecreatedunder/var/www/.Onefileisnamedindex.htmlanditscontentcontainsonelineoftextthisismyhome.Anotherfileisnamedhello.htmlanditscontentcontainsonelineoftexthelloworld.Againbeawareofthefiles'privilegesandmakesuretheyarereadablebyNginxworkerprocess.
Nowwesendrequeststothefiles'correspondingURI:
$curl'http://localhost:8080/index.html'
thisismyhome
$curl'http://localhost:8080/hello.html'
helloworld
Aswecansee,thecreatedfilecontentsaresentasoutputs.
Wecanexaminewhatishappeninghere:location/doesnothaveanycommandtoexecuteinphasecontent,thereforenomodulehasregistereda"contenthandler"inthelocation.Thehandlingthusfallstothethreestaticresourcemoduleswhicharethelastresortsofphasecontent.Theformertwomodulesngx_indexandngx_autoindexnoticesthattherequestURIdoesnotendwith/sotheyhandoffimmediatelytomodulengx_static,whichrunsintheend.Accordingtothe"documentroot"specifiedbycommandroot,modulengx_staticmapstherequestURIs/index.htmland/hello.htmltodiskfiles/var/www/index.htmland/var/www/hello.htmlrespectively.Asbothfilescanbefound,theircontentareoutputtedasresponse,meanwhileresponseheaderContent-Type,Content-LengthandLast-Modifiedareaccordinglyindicated.
Toverifymodulengx_statichasexecuted,wecouldenablethe"debuglog"introducedin(01).Againwesendrequestto/index.htmlandNginxerrorlogwillcontainfollowingdebuginformation:
[debug]3033#0:*1httpstaticfd:8
Thislineisgeneratedbymodulengx_static.Itsmeaningis"outputtingstaticresourcewhosefilehandleis8".Ofcoursethenumericalfilehandlechangeseverytime,andthelineisonlyatypicaloutputinmysetup.Tobereminded,builtinmodulengx_gzip_staticcouldgeneratethesamedebuginfoaswell,bydefaultitisnotenabledthough,whichwillbediscussedlater.
Commandrootonlydeclaresa"documentroot",itdoesnotenablesthengx_staticmodule.Themoduleisasmatteroffact,alwaysenabledalready,butitmightnothavethechancetoexecute.Thisisentirelyuptotheothermodules,whichexecuteearlierincontentphase.Modulengx_staticexecuteonlywhenallofthemhave"gaveup".Toprovethis,checkfollowingblanklocationdefinition:
location/{
}
Becausethereisnorootcommand,Nginxcomputesadefault"documentroot"whenthelocationisrequested.Thedefaultshallbethehtml/subdirectoryunder"configureprefix".Forexamplesupposeour"configureprefix"is/foo/bar/,thedefault"documentroot"is/foo/bar/html/.
Sowhodecides"configureprefix"?ActuallyittheNginxrootdirectorywhenitisinstalled(orthevalueof--prefixoptionofscript./configurewhenNginxisbuilt).IfNginxisinstalledinto/usr/local/nginx/,"configureprefix"is/usr/local/nginx/anddefault"documentroot"istherefore/usr/local/nginx/html/.Certainlyacommandlineoption--prefixcanbegivenwhenNginxisstarted,tochangethe"configureprefix"(sothatwecaneasilytestmultiplesetups).SupposeNginxisstartedasfollowing:
nginx-p/home/agentzh/test/
Forthisserver,its"configureprefix"becomes/home/agentzh/test/andits"documentroot"becomes/home/agentzh/test/html/.The"configureprefix"notonlydetermines"documentroot",itactuallydeterminesthewaymanyrelationalpathresolutestoabsolutepathinNginxconfiguration.Wewillencountermanyexampleswhichreference"configureprefix".
Infactthereisasimplewayoftellingcurrent"documentroot",whichistorequestanon-existedfile,Suchas:
$curl'http://localhost:8080/blah-blah.txt'
<html>
<head><title>404NotFound</title></head>
<bodybgcolor="white">
<center><h1>404NotFound</h1></center>
<hr><center>nginx</center>
</body>
</html>
Naturally,the404errorpageisreturned.AgainwhenwecheckNginxerrorlog,weshallhavefollowingerrormessage:
[error]9364#0:*1open()"/home/agentzh/test/html/blah-blah.txt"failed(2:Nosuchfileordirectory)
Theerrormessageisprintedbymodulengx_static,sinceitcannotfindafileblah-blah.txtinitscorrespondingpath.Andbecausetheerrormessagecontainstheabsolutepath,whichngx_staticattemptstoopenwith,it'squiteobviousthatcurrent"documentroot"is/home/agentzh/test/html/.
Manynewbiesmighttakeitforgrantedthaterror404iscausedwhentheneededlocationdoesnotexist.Theformerexampletellsus,404errorcouldbereturnedeveniftheneededlocationisconfiguredandmatched.Thisisbecauseerror404meansthenon-existenceofanabstract"resource",notthespecificlocation.
Anotherfrequentmistakeismissingthecommandforphasecontent,whentheyactuallydon'texpectthedefaultstaticmodulestocomeintoplay,forexample:
location/auth{
access_by_lua'
--alotofLuacodeomittedhere...
';
}
Apparently,onlycommandsforphaseaccessaregivenfor/auth,whichisaccess_by_lua.Andithasnocommandsforphasecontent.Sowhen/authisrequested,theLuacodespecifiedinaccessphasewillexecute,thenthestaticresourcewillbeservedinphasecontentbymodulengx_static.Sinceitactuallylooksforthefile/authonthedisknormallyitdumpsa404errorunlessweareluckilyandfile/authiscreatedonthecorrespondingpath.Sothethumbofrule,whenerror404isencounteredundernostaticresourcecircumstances,weshallfirstcheckifthelocationhasproperlyconfigureditscommandsforphasecontent,thecommandscanbecontent_by_lua,echoandproxy_passetc.Infact,Nginxerrorlogerror.logcouldonlygiveveryconfusingmessageforthecase.Astheonesbelow,whichisfoundfortheaboveexample:
[error]9364#0:*1open()"/home/agentzh/test/html/auth"failed(2:Nosuchfileordirectory)
Nginxdirectiveexecutionorder(08)Sofarwehaveaddressedindetailrewrite,accessandcontent,whicharealsothemostfrequentlyencounteredphasesinNginxrequestprocessing.WehavelearntmanyNginxmodulesandtheircommandsthatexecuteinthosephases,andit'scleartousthatthecommands'executionorderisdirectlydecidedbythephasetheyarerunningin.UnderstandingthephaseisourkeynoteforcorrectconfigurationwhichorchestratesvariousNginxmodules.Thereforelet'scovertherestphaseswe'venotmet.
Asmentionedin(01),altogethertherecanbe11phaseswhenNginxhandlesarequest.Intheirexecutionorderthephasesarepost-read,server-rewrite,find-config,rewrite,post-rewrite,preaccess,access,post-access,try-files,content,andfinallylog.
Phasepost-readistheveryfirst,commandsregisteredinthisphaseexecuterightafterNginxhasprocessedtherequestheaders.Similartophaserewritewe'velearntearlier,post-readsupportshooksbyNginxmodules.Built-inmodulengx_realipisanexample,ithooksitshandlerinpost-readphase,andforcefullyrewritetherequest'soriginaladdressasthevalueofaspecificrequestheader.Thefollowingcaseillustratesngx_realipmoduleanditscommandsset_real_ip_from,real_ip_header.
server{
listen8080;
set_real_ip_from127.0.0.1;
real_ip_headerX-My-IP;
location/test{
set$addr$remote_addr;
echo"from:$addr";
}
}
TheconfigurationtellsNginxtoforcefullyrewritetheoriginaladdressofeveryrequestcomingfrom127.0.0.1tobethevalueoftherequestheaderX-My-IP.Meanwhileitusesthebuilt-invariable$remote_addrtooutputtherequest'soriginaladdress,sothatweknowiftherewriteissuccessful.
Firstwesendarequestto/testfromlocalhost:
$curl-H'X-My-IP:1.2.3.4'localhost:8080/test
from:1.2.3.4
Thetestutilizes-Hoptionprovidedbycurl,theoptionincorporatesanextraHTTPheaderX-My-IP:1.2.3.4intherequest.Aswecantell,variable$remote_addrhasbecome1.2.3.4inrewritephase,thevaluecomesfromtherequestheaderX-My-IP.SowhendoesNginxrewritetherequest'soriginaladdress?yesit'sinthepost-readphase.Sincephaserewriteisfarbehindphasepost-read,whencommandsetreadsvariable$remote_addr,itsvaluehasalreadybeenrewritteninpost-readphase.
Ifhowever,therequestsentfromlocalhostto/testdoesnothaveaX-My-IPheaderortheheadervalueisaninvalidIPaddress,Nginxwillnotmodifytheoriginaladdress.Forexample:
$curllocalhost:8080/test
from:127.0.0.1
$curl-H'X-My-IP:abc'localhost:8080/test
from:127.0.0.1
Ifarequestissentfromanothermachineto/test,itoriginaladdresswon'tbeoverwrittenbyNginxeither,evenifithasaperfectX-My-IPheader.Itisbecauseourpreviouscasemarksexplicitlywithcommandset_real_ip_from,thattherewritingonlyoccursfortherequestscomingfrom127.0.0.1.ThisfilteringmechanismprotectNginxfrommaliciousrequestssentbyuntrustedsources.Asyoumighthaveexpected,commandset_real_ip_fromcandesignateaIPsubnet(byusingCIDRnotationintroducedearlierin(03)).Besides,commandset_real_ip_fromcanbeusedmultipletimessothatwecansetupmultipletrustedsources,belowisanexample:
set_real_ip_from10.32.10.5;
set_real_ip_from127.0.0.0/24;
Youmightbeasking,what'sthebenefitmodulengx_realipbringstous?Whywouldwerewritearequest'soriginaladdress?Theansweris:whentherequesthascomethroughoneormoreHTTPproxies,themodulebecomesveryhandy.Whenarequestisforwardedbyaproxy,itsoriginaladdresswillbecometheproxyserver'sIPaddress,consequentlyNginxandtheservicesrunningonitwillnolongerhavetheactualsource.However,wecouldletproxyserverrecordtheoriginaladdressinaspecificheader(suchasX-My-IP)andrecoveritinNginx,sothatitssubsequentprocessing(andtheservicesrunningonNginx)willtaketherequestasifitcomesrightfromitsoriginaladdressandtheproxiesinbetweenaretransparent.Forthisexactpurpose,modulengx_realipneedshookhandlersinthefirstphase,thepost-readphase,sotherewritingoccursasearlyaspossible.
Behindpost-readistheserver-rewritephase.Webrieflymentionedin(02),whenmodulengx_rewriteanditscommandsareconfiguredinserverdirective,theybasicallyexecuteinserver-rewritephase.Wehaveanexamplebelow:
server{
listen8080;
location/test{
set$b"$a,world";
echo$b;
}
set$ahello;
}
Attentiontheset$ahellostatementisputinserverdirective,soitrunsinserver-rewritephase,whichrunsearlierthanrewritephase.Thereforestatementset$b"$a,world'"inlocationdirectiveisexecutedafterwardsanditobtainsthecorrect$avalue:
$curllocalhost:8080/test
hello,world
Sincephaseserver-rewriteexecuteslaterthanpost-readphase,commandsetinserverdirectivealwaysrunslaterthanmodulengx_realip,whichrewritestherequest'soriginaladdress,example:
server{
listen8080;
set$addr$remote_addr;
set_real_ip_from127.0.0.1;
real_ip_headerX-Real-IP;
location/test{
echo"from:$addr";
}
}
Sendrequestto/testwehave:
$curl-H'X-Real-IP:1.2.3.4'localhost:8080/test
from:1.2.3.4
Again,commandsetiswritteninfrontofcommandsofngx_realip,itsactualexecutionisonlyafterwards.Sowhencommandsetassignsvariable$addrinserver-rewritephase,thevariable$remote_addrhasbeenoverwritten.
Nginxdirectiveexecutionorder(09)Rightafterserver-rewriteisthephasefind-config.ThisphasedoesnotallowNginxmodulestoregistertheirhandlers,insteaditisaphasewhenNginxcorematchesthecurrentrequesttothelocationdirectives.Itmeansarequestisnotcateredbyanylocationdirectiveuntilitreachesfind-config.Apparently,forphaseslikepost-readandserver-rewrite,theeffectivecommandsarethosewhichgetspecifiedonlyinserverdirectivesandtheirouterdirectives,becausethetwophasesareexecutedearlierthanfind-config.Thisexplainsthatcommandsofmodulengx_rewriteareexecutedinphaseserver-rewriteonlyiftheyarewrittenwithinseverdirective.Similarly,theformerexamplesconfigurethecommandsofmodulengx_realipinserverdirectivetomakesurethehandlersregisteredinpost-readphasecouldfunctioncorrectly.
AssoonasNginxmatchesalocationdirectiveinthefind-configphase,itprintsadebuglogintheerrorlogfile.Let'scheckfollowingexample:
location/hello{
echo"helloworld";
}
IfNginxenablesthe"debuglog",adebuglogcanbecapturedinfileerror.logwheneverinterface/helloisrequested.
$grep'usingconfig'logs/error.log
[debug]84579#0:*1usingconfiguration"/hello"
Forthepurposeofconvenience,thelog'stimestamphasbeenomitted.
Afterphasefind-config,itisouroldbuddyrewrite.SinceNginxalreadymatchestherequesttoaspecificlocationdirective,startingfromthisphase,commandswrittenwithinlocationdirectivesarebecomingeffective.Asillustratedearlier,commandsofmodulengx_rewriteareexecutedinrewritephasewhentheyarewritteninlocationdirectives.Likewise,commandsofmodulengx_set_miscandmodulengx_lua(set_by_luaandrewrite_by_lua)arealsoexecutedinphaserewrite.
Afterrewrite,itisthepost-rewritephase.Justlikefind-config,thisphasedoesnotallowNginxmodulestoregistertheirhandlerseither,insteaditcarriesouttheneeded"internalredirects"byNginxcore(ifthishasbeenrequestedinrewritephase).Wehaveaddressedthe"internaljump"conceptin(02),anddemonstratedhowtoissuethe"internalredirect"withcommandecho_execorcommandrewrite.However,let'sfocusoncommandrewriteforthemomentsincecommandecho_execisexecutedincontentphaseandbecomesirrelevanttopost-rewrite,theformerdrawsgreaterinterestbecauseitexecutesinrewritephase.Backtoourexamplein(02):
server{
listen8080;
location/foo{
set$ahello;
rewrite^/bar;
}
location/bar{
echo"a=[$a]";
}
}
Thecommandrewritefoundindirectivelocation/foo,rewritestheURIofcurrentrequestas/barunconditionally,meanwhile,itissuesan"internalredirect"andexecutioncontinuesfromlocation/bar.Whatultimatelyintriguesus,isthemagicalbitsandpiecesof"internalredirect"mechanism,"internalredirect"effectivelyrewindsourprocessingofcurrentrequestbacktothefind-configphase,sothatthelocationdirectivescanbematchedagaintotherequestURI,whichusuallyhasbeenrewritten.Justlikeourexample,whoseURIisrewrittenas/barbycommandrewrite,thelocation/bardirectiveismatchedandexecutionrepeatstherewritephasethereafter.
Itmightnotbeobvious,thattheactualactofrewindingtofind-configdoesnotoccurinrewritephase,insteaditoccursinthefollowingpost-rewritephase.Commandrewriteintheformerexample,simplyrequestsNginxtoissuean"internalredirect"initspost-rewritephase.ThisdesignisusuallyquestionedbyNginxbeginnersandtheytendtocomeupwithanideatoexecutethe"internaljump"directlybycommandrewrite.Theanswerhowever,isfairlysimple.ThedesignallowsURIberewrittenmultipletimesinthelocationdirective,whichismatchedattheverybeginning.Suchas:
location/foo{
rewrite^/bar;
rewrite^/baz;
echofoo;
}
location/bar{
echobar;
}
location/baz{
echobaz;
}
TherequestURIhasbeenrewrittentwiceinlocation/foodirective:firstlyitbecomes/bar,secondlyitbecomes/baz.Astheneteffectofbothrewritestatements,"internalredirect"occursonlyonceinpost-rewritephase.Ifitwouldhaveexecutedthe"internalredirect"atthefirstURIrewrite,thesecondwouldhavenochancetobeexecutedsinceprocessingwouldhaveleftcurrentlocationdirective.Toprovethiswesendarequestto/foo:
$curllocalhost:8080/foo
baz
Itcanbeassertedfromtheoutput,theactualjumpisfrom/footo/baz.WecouldfurtherprovethisbyenablingNginx"debuglog"andinterrogatethedebuglog
generatedinfind-configphaseforthematched:
$grep'usingconfig'logs/error.log
[debug]89449#0:*1usingconfiguration"/foo"
[debug]89449#0:*1usingconfiguration"/baz"
Clearly,forthespecificrequest,Nginxonlymatchestwolocationdirectives:/fooand/baz,and"internaljump"occursonlyonce.
Quiteobviously,ifcommandngx_rewrite/rewriteisusedtorewritetherequestURIinserverdirective,therewon'tbeany"internalredirects",thisisbecausetheURIrewriteishappeninginserver-rewritephase,whichgetsexecutedearlierthanfind-configphasethatmatchesinbetweenthelocationdirectives.Wecanchecktheexamplebelow:
server{
listen8080;
rewrite^/foo/bar;
location/foo{
echofoo;
}
location/bar{
echobar;
}
}
Intheexample,everyrequestwhoseURIstartswith/foogetsitsURIrewrittenas/bar.Therewritingoccursinserver-rewritephase,andtherequesthasneverbeenmatchedtoanylocationdirective.OnlyafterwardsNginxexecutesthematchesinfind-configphase.Soifwesendarequestto/foo,location/foonevergetsmatchedbecausewhenthematchoccursinfind-configphase,therequestURIhasbeenrewrittenas/bar.Solocation/baristheoneandtheonlyonematcheddirective.Actualoutputillustratesthis:
$curllocalhost:8080/foo
bar
Againlet'scheckNginx"debuglog":
$grep'usingconfig'logs/error.log
[debug]92693#0:*1usingconfiguration"/bar"
Aswecantell,Nginxaltogetherfinishesoncethelocationmatch,andthereisno"internalredirect".
Nginxdirectiveexecutionorder(10)Afterpost-rewrite,itisthepreaccessphase.Justasitsnameimplies,thephaseiscalledpreaccesssimplybecauseitisexecutedrightbeforeaccessphase.
Built-inmodulengx_limit_reqandngx_limit_zoneareexecutedinthisphase.Theformerlimitsthenumberofrequestsperhour/minute,andthelatterlimitsthenumberofsimultaneousrequests.Wewillbediscussingthemmorethoroughlyafterwards.
Actually,built-inmodulengx_realipregistersitshandlerinpreaccessaswell.Youmightneedtoaskthen:"whydoitagain?Diditregisteritshandlersinpost-readphasealready".Beforetheanswerisuncoveredlet'sstudyfollowingexample:
server{
listen8080;
location/test{
set_real_ip_from127.0.0.1;
real_ip_headerX-Real-IP;
echo"from:$remote_addr";
}
}
Comparingtotheearlierexample,themajordifferenceisthatcommandsofmodulengx_realiparewritteninaspecificlocationdirective.Aswehavelearntbefore,Nginxmatchesitslocationdirectivesinfind-configphase,whichisfarbehindpost-read,hencetherequesthasnothingtodowithcommandswritteninanylocationdirectiveinpost-readphase.Backtoourexample,itisexactlythecasewherecommandsarewritteninalocationdirectiveandmodulengx_realipwon'tcarryoutanyrewriteoftheremoteaddress,becauseitisnotinstructedassuchinpost-readphase.
Whatifwedoneedtherewrite?Tohelpresolvetheissue,modulengx_realipregistersitshandlersinpreaccessagain,sothatitisgiventhechancetoexecuteinalocationdirective.Nowtheexamplerunsaswewould'veexpected:
$curl-H'X-Real-IP:1.2.3.4'localhost:8080/test
from:1.2.3.4
Bereallycarefulthough,modulengx_realipcouldeasilybemisused,asourfollowingexampleillustrates:
server{
listen8080;
location/test{
set_real_ip_from127.0.0.1;
real_ip_headerX-Real-IP;
set$addr$remote_addr;
echo"from:$addr";
}
}
Intheexample,weintroducesavariable$addr,towhichthevalueof$remote_addrissavedinrewritephase.Thevariableisthenusedintheoutput.Slowdownrighthereandyoumighthavenoticedtheissue,phaserewriteoccursearlierthanpreaccess,sovariableassignmentactuallyhappensbeforemodulengx_realiphasthechancetorewritetheremoteaddressinpreaccessphase.Theoutputprovesourobservation:
$curl-H'X-Real-IP:1.2.3.4'localhost:8080/test
from:127.0.0.1
Theoutputgivestheactualremoteaddress(nottherewrittenone)AgainNginx"debuglog"helpsassertittoo:
$grep-E'httpscript(var|set)|realip'logs/error.log
[debug]32488#0:*1httpscriptvar:"127.0.0.1"
[debug]32488#0:*1httpscriptset$addr
[debug]32488#0:*1realip:"1.2.3.4"
[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F
[debug]32488#0:*1httpscriptvar:"127.0.0.1"
Amongthelogs,thefirstlinewrites:
[debug]32488#0:*1httpscriptvar:"127.0.0.1"
Thelogisgeneratedwhenvariable$remote_addrisfetchedbycommandset,string"127.0.0.1"isthefetchedvalue.
Thesecondlinewrites:
[debug]32488#0:*1httpscriptset$addr
ItindicatesNginxassignsvaluetovariable$addr.
Forthefollowingtwolines:
[debug]32488#0:*1realip:"1.2.3.4"
[debug]32488#0:*1realip:0100007FFFFFFFFF0100007F
Theyaregeneratedwhenmodulengx_realiprewritestheremoteaddressinpreaccessphase.Aswecantell,thenewaddressbecomes1.2.3.4asexpectedbutithappensonlyafterthevariableassignmentandthat'salreadytoolate.
Nowthelastline:
[debug]32488#0:*1httpscriptvar:"127.0.0.1"
Itisgeneratedwhencommandechooutputsvariable$addr,clearlythevalueistheoriginalremoteaddress,nottherewrittenone.
Somepeoplemightcomeupwithasolutionimmediately:"whatifmodulengx_realipregistersitshandlersinrewritephaseinstead,notinpreacccessphase?"Thesolutionhoweveris,notnecessarilycorrect.Thisisbecausemodulengx_rewriteregistersitshandlersinrewritephasetoo,andwehavelearntin(02)thattheexecutionorder,underthecircumstances,cannotbeguaranteed,sothereisagoodchancethatmodulengx_realipstillexecutesitscommandsaftercommandset.
Alwayswehavethebackupoption:insteadofpreaccess,tryusengx_realipmoduleinserverdirective,itbypassesthebothersomesituationsencounteredabove.
Afterphasepreaccess,itisanotheroldfriend,theaccessphase.Aswe'velearnt,built-inmodulengx_access,3rdpartymodulengx_auth_requestand3rdpartymodulengx_lua(access_by_lua)havetheircommandsexecutedinthisphase.
Afterphaseaccess,itisthepost-accessphase.Againasthenameimplies,wecaneasilyspotthatthephaseisexecutedrightafteraccessphase.Similartopost-rewrite,thephasedoesnotallowNginxmoduletoregistertheirhandlers,insteaditrunsafewtasksbyNginxcore,amongthem,primarilyisthesatisfyfunctionality,providedbymodulengx_http_core.
WhenmultipleNginxmoduleexecutetheircommandsinaccessphase,commandsatisfycontrolstheirrelationshipsinbetween.Forexample,bothmoduleAandmoduleBregistertheiraccesscontrolhandlersinaccessphase,wemayhavetwoworkingmodes,oneistoletaccesswhenbothAandBpasstheircontrol,theotheristoletaccesswheneitherAorBpasstheircontrol.Thefirstoneiscalledallmode("AND"relation),thesecondoneiscalledanymode("OR"relation)Bydefault,Nginxusesallmode,belowisanexample:
location/test{
satisfyall;
denyall;
access_by_lua'ngx.exit(ngx.OK)';
echosomethingimportant;
}
Under/testdirective,bothngx_accessandngx_luaareused,sowehavetwomodulesmonitoringaccessinaccessphase.Specifically,statementdenyalltellsmodulengx_accesstorejectsallaccess,whereasstatementaccess_by_lua'ngx.exit(ngx.OK)'allowsallaccess.Whenallmodeisusedwithcommandsatisfy,itmeanstoletaccessonlyifeverymoduleallowsaccess.Sincemodulengx_accessalwaysrejectsinourcase,therequestisrejected:
$curllocalhost:8080/test
<html>
<head><title>403Forbidden</title></head>
<bodybgcolor="white">
<center><h1>403Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
CarefulreadersmightfindfollowingerrorlogintheNginxerrorlogfile:
[error]6549#0:*1accessforbiddenbyrule
Ifhowever,wechangethesatisfyallstatementtosatisfyany.
location/test{
satisfyany;
denyall;
access_by_lua'ngx.exit(ngx.OK)';
echosomethingimportant;
}
Theoutcomeiscompletelydifferent:
$curllocalhost:8080/test
somethingimportant
Therequestisallowedtoaccess.Becauseoverallaccessisallowedwheneveronemodulepassesthecontrolinanymode.Inourexample,modulengx_luaanditscommandaccess_by_luaalwaysallowtheaccess.
Certainly,ifeverymodulerejectstheaccessinthesatisfyanycircumstances,therequestwillberejected:
location/test{
satisfyany;
denyall;
access_by_lua'ngx.exit(ngx.HTTP_FORBIDDEN)';
echosomethingimportant;
}
Nowrequestto/testwillencounter403Forbiddenerrorpage.Intheprocess,the"OR"relationofaccesscontrolofeachaccessmodule,isimplementedinpost-access.
Pleasenotethatthisexamplerequiresatleastngx_lua0.5.0rc19orlater;earlierversionscannotworkwiththesatisfyanystatement.