W H I T E P A P E R

SD-WAN Overlays, Topologies, and

Deployment Models

TopologiesThis white paper discusses various SD-WANOverlayOptionsandtheirconfigurations.Differentoverlaytypes areneededtoconnecttoprivate(MPLS)vspublic(internet)networks.

Usingthese overlays,thevarioustopologiesarecreatedasexplainedintheOverlayTopologies section. The different hosting options available for the SD-WANcomponents are explained in the Hosting Variants section. Site Topology andRedundancyare subsequentlycovered.

2101924

ContentsOverlay OptionsOverlay TopologiesDeployment ModelsSite Topology and Redundancy

2

OverlayOptions

The VMware SD-WAN Edge uses interfaces that can be used to simultaneouslyroutetotheunderlayandestablishSD-WANOverlay.Asingleroutedinterfacecanalso have multiple public and/or private overlays allocated to it, separated by802.1qVLANtags.

Theunderlayistheexistingnetworkinfrastructureandcanbeeitherpublic(internet)orprivate(MPLS).AnoverlayisbuiltontopofanunderlayutilizingVCMPtunnels.

PublicandPrivateOverlays

ApublicWANoverlay isdefinedasonethatrunsoverapublicunderlaynetworkwhere a VMware SD-WAN Gateway is reachable. A public WAN overlay isautomatically detectedand createdoneachedgewhenan interface to apublicnetwork comes up, and after a successful negotiation with a VMware SD-WANGateway.

A SD-WAN overlay can be configured to be auto-detected or user-defined. Bydefault, all routed interfaces are configured to auto-detect the overlay on thatinterfacewhenacable is inserted.Asseen inFigure4.1, this isdonebysendingouta"tunnelinitiationmessage"toalistofassignedVMwareSD-WANGateways.

Figure4.1:VMwareSD-WANEdgetunnelsetup

A valid response from a VMware SD-WAN Gateway and subsequent successfulbandwidth test (or amanual definition of available bandwidth) will result in thesetupofVCMPtunnels fromtheVMwareSD-WANEdgetooneormoreVMwareSD-WAN Gateways and this newly discovered WAN overlay is reported to theVMwareSD-WANOrchestrator.Thisbehaviorisdeterminedbythe Auto-DetectOverlay menu option under Configure > Edges > {Edge} > Devicetab > {Interface_number} (see Figure 4.2). ThePublic IP of the overlay isderived from the tunnel information exchanged with the VMware SD-WANGateway.

Figure4.2:AroutedinterfaceonanVMwareSD-WANEdgewiththedefaultWANoverlaysettingapplied

Conversely, a privateWAN overlay is user-defined and is carried over a privatenetworkwhereaVMwareSD-WANGatewayisnotreachable.TheIPaddressoftheinterface associatedwith a user-defined private overlay is used to populate thepeer tableof other VMware SD-WAN Edges in the enterprise. This provides thenecessarytunneldestinationinformationrequiredtoinitiateandmaintaintunnelsonthisprivateunderlaynetwork.

For interfaces attached to a private underlay, the setting for WAN Overlay inConfigure > Edge > {edge name} > Device tab > {interface}mustbesetto user-defined Overlay .ThiswillinstructtheedgetopasstheinterfaceIPaddressassociatedwiththeuser-definedoverlay,akaprivateaddress,to the VMwareSD-WANOrchestrator. TheVMware SD-WANOrchestratorwill inturnpassthisprivateaddresstootherVMwareSD-WANEdgesthatareconfiguredforauser-definedoverlay,sothoseremoteVMwareSD-WANEdgescanlearnthetunnelendpointIPaddressovertheprivatenetwork,neededwhenbuildinganSD-WANoverlay.Figure4.3showsanexampleofaninterfaceconfiguredfora user-defined overlay .

Conversely, a privateWAN overlay is user-defined and is carried over a privatenetworkwhereaVMwareSD-WANGatewayisnotreachable.TheIPaddressoftheinterface associatedwith a user-defined private overlay is used to populate thepeer tableof other VMware SD-WAN Edges in the enterprise. This provides thenecessarytunneldestinationinformationrequiredtoinitiateandmaintaintunnelsonthisprivateunderlaynetwork.

For interfaces attached to a private underlay, the setting for WAN Overlay inConfigure > Edge > {edge name} > Device tab > {interface}mustbesetto user-defined Overlay .ThiswillinstructtheedgetopasstheinterfaceIPaddressassociatedwiththeuser-definedoverlay,akaprivateaddress,to the VMwareSD-WANOrchestrator. TheVMware SD-WANOrchestratorwill inturnpassthisprivateaddresstootherVMwareSD-WANEdgesthatareconfiguredforauser-definedoverlay,sothoseremoteVMwareSD-WANEdgescanlearnthetunnelendpointIPaddressovertheprivatenetwork,neededwhenbuildinganSD-WANoverlay.Figure4.3showsanexampleofaninterfaceconfiguredfora user-defined overlay .

3

4

Figure4.3:VMwareSD-WANEdgeinterfaceconnectedtoaprivatenetworkwithuser-definedoverlayselected

T E C H T I P

ForinterfacesthatdonotneedaWANoverlay,e.g.facingtheLAN,uncheck

the WAN Overlay checkbox .ThisinterfacewilljusthandletrafficdirecttoLAN-sideclientsandservers.

Oncean interfaceisconfiguredforauser-definedWANOverlay,youthensimplycreate a new user-defined WAN Overlay and bind it to the interface alreadyconfigured foruser-definedoverlay asper above. For example, in Figure4.4weshow the user-defined WAN Overlay configuration available underConfigure > Edge > {edge name} > Device tab > Add user-defined WAN Overlay and Link Type issetto Private .

5

Figure4.4:Newuser-definedWANoverlaywithalinktypeofprivate,beingboundto GE2 whichhasalreadybeensettocarryauser-definedoverlay

There is an optional configuration within the user-defined WAN overlayconfiguration todefinetheIPaddress,next-hopIP,andVLANID,otherwiseitwillinheritthosepropertiesdirectlyfromtheinterface.

T E C H T I P

AlwayssettheWANoverlayto user-defined forprivatenetworks.Ifthe

WANoverlayisleftas auto-detect overlay, andahostedVMwareSD-WANGatewayisreachableviatheMPLSnetwork(typicallyviaadatacenter

firewall),theVMwareSD-WANEdgewillincorrectlycreateaWANoverlayof

type"public"viaaprivatenetwork,and,inthiscase,thepublicaddressofthe

firewallinthedatacenterwilllikelybecomethetunnelendpointaddressforthis

VMwareSD-WANEdge,whichisnotdesirable.

SD-WANServiceReachableUnderprivateoverlays,thereisanoptiontoenableafeatureknownas SD-WANService Reachable . Selecting thisoption informs theVMwareSD-WANEdgethatthereexistsontheprivatenetworkameanstobreakouttothecloud-hostedVMwareSD-WANGatewaysandOrchestrator.Whenselectingthisoption,alistofpublic/32addresseswillbeshownthatrepresentsthecloud-hostedVMwareSD-WAN Gateways and Orchestrator. A network engineer should advertise thoseprefixes into the private network, in case advertising a default route into theprivatenetworkisnotdesirable.

6

T E C H T I P

Alwaysenablethe SD-WAN Service Reachable settingwhentheprivatenetworkprovidesreachabilitytotheinternet,asthisfeatureprovidesanSD-

WANOverlayfailoverpathtocloud-hostedVMwareSD-WANGatewaysand

Orchestratorinthecasewherethelinkstothepublicinternetfail.

SettingaPrivateNetworkNameInsomecases,VMwareSD-WANEdgeinthedatacentermaybetheaggregationpoint for multiple MPLS provider networks e.g. MPLS A and MPLS B. In thisscenario,allVMwareSD-WANEdgesconnectedtoMPLSAshouldnotattempttobuildtunnelstotheprivateIPaddressesofVMwareSD-WANEdgesconnectedtoMPLS B. To help with this, the VMware SD-WAN solution allows the networkengineertoexplicitlynameeachprivatenetworkandassignthenametoindividualprivateWANoverlaystoensuretunnelinitiationonlyhappenswithinthecommonprivate network. As shown in Figure 4.5 a network engineer can navigate toPrivate Network Names andcreatenewentriesbygoingto Configure >Network Services > Private Network Name > {New} > {PrivateNetwork Name} > {Save Changes}

Figure4.5:Createanewnameforusewithaspecificprivatenetwork

OneOverlayperPhysicalInterface

Figure 4.6 shows a typical hybrid WAN branch. The VMware SD-WAN EdgeterminatestwoWANinterfaces,onepublicandoneprivate.NotetheVMwareSD-WAN Edge can terminate the MPLS link from the PE directly (copper or fiberEthernet).Inthisscenario,thereisaone-to-onerelationshipbetweenthephysicalinterfaceandWANoverlay.Thephysicalinterfaceitselfcanhave802.1QVLANtagaswell.

7

Figure4.6:Oneoverlayperphysicalinterface

MultipleOverlaysperPhysicalInterface

Figure4.7 showsanother scenariowhere theVMwareSD-WANEdge terminatesonlyonephysicalinterface.Theupstreamswitchportisconfiguredastrunktopassmultiple VLANs with different 802.1Q tags to the VMware SD-WAN Edge. Someservice providersmay choose to deliver bothMPLS and internet over the samephysicalinterface.TheinterfaceisconfiguredwiththeIPaddressofthefirstVLAN(101), and the next-hop information to reach the internet. Next create a publicuser-definedWANoverlaywiththesameIPaddress,next-hopandVLANID.Thencreateaprivateuser-definedWANoverlayanduse theoptional configuration intheWANoverlayfortheprivatenetworkIPaddress,next-hopandVLANID(102).Figure4.8showsascreenshotofthepublicuser-definedWANoverlay.Figure4.9showsascreenshotoftheprivateuser-definedWAN.

Figure4.7:Multipleoverlaysperphysicalinterface

8

Figure4.8:Publicuser-definedWANOverlay

Figure4.9:Privateuser-definedWANOverlay

9

KeyTakeaways:

• Bydefault,allroutedinterfacesareconfiguredtoauto-detectthetunnel

• SD-WANoverlayonprivatelinksshouldbeconfiguredasuser-defined

• DisableWANoverlaysettingsonLAN-facingroutedinterfaces• VMwareSD-WANsupportsmultipleSD-WANoverlayonasinglerouted

interface

• Enable the SD-WAN Service Reachable feature for edgeswithprivate links for reachability to VMware SD-WAN Orchestrator andVMwareSD-WANGatewayonpublicinternet

• To separateWANoverlaysona setupwithprivate links fromdifferentISPs,usetheprivatenetworknameoption

10

OverlayTopologies

Overlay topologiesare formedusing theoverlay typesdiscussed in thepreviouschapter.This chapterwill coverdifferent topology types suchas traditionalhub-and-spokeorbranch-to-branch.

The SD-WAN overlay can be built over any network that supports routing of IPpackets, such as wired internet services, wireless internet services, privatenetworks such as MPLS, along with point-to-point services such as those overradio,microwave,orfibre.SD-WANoverlaysarebuiltusingVCMPtunnels.

By default, all VMware SD-WAN Edges in a VMware SD-WAN solution will buildVCMPtunnelstoadesignatedprimaryandsecondaryVMwareSD-WANGateway.In addition, the SD-WAN overlay can be configured to buildBranch to BranchVCMP tunnels and additional VMware SD-WAN Edge to VMware SD-WANGatewayVCMPtunnelsforaccesstoNon-VeloCloudSites.ThisisaccomplishedusingIPSectunnelsfromVMwareSD-WANGatewaytoIPSectunnelend-points.ANon-VeloCloudSiteisanygenericIPSecdestination.

ANon-VeloCloudSitecanbeanygenericIPSecdestination,whichshouldbemadeaccessible from VMware SD-WAN Edges. This is accomplished using an IPSectunnelfromtheVMwareSD-WANGatewaytotheconfiguredNon-VeloCloudSite.

TheVCMPprotocolworksthroughNATandimplementsaproprietaryheaderforpacketsequencing,timestamping,anddetectingpacketloss.

Inordertocreatetopologies,anetworkengineerwillusetheCloudVPNfeature.Figure 4.10 from the VMware SD-WANOrchestrator User Interface provides anexample of how to prepare VMware SD-WAN Edges in the sameprofile forSD-WAN overlay beyond the default VMware SD-WAN Edge to VMware SD-WANGateway tunnels, by using theCloud VPN slider. This action, performed at theprofile level in the VMware SD-WANOrchestrator, instructs all VMwareSD-WANEdgesinthesameprofiletoenablecontrolplanerouting.ToreachthissectionintheVMwareSD-WANOrchestratorUI,navigate to Configure > Profiles >{Profile Name} > Device Tab > Cloud VPN

11

Figure4.10:VMwareSD-WANOrchestratorUI—CloudVPNsliderforenablingcontrolplanerouting

This base level cloud VPN configuration is typically applied at the profile forVMwareSD-WANEdgesdeployedintothedatacenter(DC).

AHub is an explicit role that's assigned to a VMware SD-WAN Edge. Hubs aretypicallylocatedinthedatacenterandtheyterminatemultipleoverlaytunnels.IntheprofileforaHub,branch-to-branchVPNistypicallynotenabledasservers ineach data center will leverage the data center interconnect (DCI) forcommunicationratherthanrelyingonVMwareSD-WANEdgesineachDCtobuildtunnelstoeachother.

AseparateprofileisrecommendedforbranchVMwareSD-WANEdgeswhichwillpointtotheHubsviathe Branch to VeloCloud Hubs configuration.

Hub-and-SpokeVPN

A VMware SD-WAN hub-and-spoke topology describes one or more branchVMwareSD-WANEdgesakaspokes,buildsoneormorepermanenttunnelstoacentrallylocatededgeinheadofficeorthedatacenter—knownastheHub.Figure4.11illustratesatypicalhub-and-spoketopologywithoneVMwareSD-WANEdgeconfiguredasaHub.

12

Figure4.11:VMwareSD-WAN—exampleofhub-spoketopology

Configuration for nominating a hub is typically done in the VMware SD-WANOrchestrator using aprofilewith branch sites asmembers. In the sameprofile,Branch to VeloCloud Hubs isenabledandVMwareSD-WANEdgesthataretobehubsforthebranchesinthisprofileareselectedfromalist.SeeFigure4.12foranexamplewheretwodatacenteredgesarebeingselectedashubs.Navigateto Configure > Profiles > {Profile Name} > Device Tab > CloudVPN > Branch to VeloCloud Hubs > Enable

This configurationwill instruct all VMware SD-WANEdges that are amemberofthis profile to build permanent tunnels to DataCenter1 and DataCenter2VMwareSD-WANEdges.

13

Figure4.12:VMwareSD-WANOrchestratorprofilewithhubselection

BranchtoBranchVPN

VMware SD-WAN can be configured to allow branch-to-branch communication.Thetrafficpathforbranch-to-branchcaneitherbevia:

• VMwareSD-WANGateways(selectedbydefault);or

• VMwareSD-WANEdgesconfiguredashubs-selectedintheprofileunderBranch to Branch VPN

• VMware SD-WAN Edges can also be configured to utilize DynamicBranch to Branch VPN ascoveredinFigure4.13

NotethatVMwareSD-WANGatewayscanbeusedtostitchtunnels together, i.e.routebetweenthem, inthesamefashionasahub.Inhybriddeploymentswhereinternet and MPLS underlays are in use, a hub is a better choice for routingbetweentunnelsasahubwilltypicallyhaveconnectivitytobothunderlaytypes.

14

Figure4.13:ExampleofBranchtoBranchVPNviaeitherahub,gateway,orDynamicBranchtoBranchVCMPTunnel

Whenyouenable Dynamic Branch to Branch VPN ,thefirstfewpacketswillgothroughthecloud-hostedVMwareSD-WANGatewayortheHub.IftheinitiatingVMware SD-WAN Edge determines that traffic can be routed branch-to-branchdirect using a VCMP tunnel between VMware SD-WAN Edges, then a directdynamictunneliscreatedbetweenthebranches.Oncethetunnelisestablished,trafficbegins to flowover theVCMP tunnel between thebranches. In theeventdynamic branch to branch tunnels fail to establish (e.g. if one of the edges isbehindsymmetricNAT),trafficwillcontinuetoflowviaVMwareSD-WANGatewayorthehub.

Figure4.14showsthe trafficpathtransitingtheHub(darkblue)or transiting thecloud-hosted VMware SD-WAN Gateway (light blue), then the branch-to-branchdirectpathifenabled(bluedottedline).Afteranidletimeout(notrafficbetweenbranches)of3minutes,theinitiatingVMwareSD-WANEdgewilltriggerateardownofthedynamictunnel.

15

Figure4.14:Trafficflowforbranch-to-branchviaeitherhubedge,gateway,ordirect

To configure branch-to-branch, the Cloud VPN service is turned on in theprofile which has branch edges as members, Branch to VeloCloudHubs areselected,andthe Branch to Branch VPN isthenchecked.Asseenin Figure 4.15, there is an option to select Cloud Gateways or VeloCloudHubs as the transit device for branch-to-branch communication. Navigate toConfigure > Profiles > {Profile Name} > Device Tab > CloudVPN > Branch to Branch VPN > Enable and > Dynamic Branch ToBranch VPN > Enable

16

Figure4.15:VMwareSD-WANOrchestratorprofilewithDynamicBranchToBranchVPNenabled

T E C H T I P

Bydefault,aVMwareSD-WANEdgedoesnotrespondtoinboundconnection

attempts.However,afterenablingthisfeature,theVMwareSD-WANEdgenow

respondsonUDPport2426.

BranchtoBranchVPNusingProfileIsolation

VMware SD-WAN can be configured to allow branch-to-branch communicationbetween branches from different profiles as long as they are connected to acommon VMware SD-WAN Gateway or Hub. If a customer wants to prevent abranchVMwareSD-WANEdgeassociatedwithagivenprofilefromconnectingtoanother branch associated with a different profile, VMware SD-WAN offersSegmentation andVPNProfile Isolation.The recommendedmethod is tousetheSD-WANsegmentation feature toassignVMwareSD-WANEdges todifferentsegments.Refer toChapter6:Security, sectionVMwareSD-WANSegmentation, formoreinformationonthisfeature.

Forcustomerswhoprefertonotenablethesegmentationfeatureorwhowanttoachieve VMware SD-WAN Edge isolation within one segment, the Isolate

17

Profile canbeused(seeFigure4.15again).

WhenVPNProfileIsolation isenabledforaprofile, theVMwareSD-WANEdgeswithinthatprofilewillonlylearn:

• RoutestoVMwareSD-WANEdgeswithinitsownprofile.

• Routes to the assigned Hubs as well as underlay routes learned by theHub.

CloudVPNforNon-VeloCloudSite

There are several situationswhere a site isnot yet able tomigrate toSD-WAN.Exampleswouldbeduringmigration,orduringamergerandacquisition,wherenoteverysitecanbemovedtoSD-WANinasingleday.Theotherexamplewouldbe a third-party provider that doesn't have VMware SD-WAN Edges installedlocally.ThesitethatdoesnotrunSD-WANiscalledaNon-VeloCloudSite(NVS).Figure4.16showsatypicaltopologyofconnectingtoaNVS.TheVMwareSD-WANGateway formsastandard IPSec tunnel to thedevice in theNVS, represented inFigure4.16bya traditionaldatacenter.FromthebranchVMwareSD-WANEdgeperspective, it formsaVCMPtunneltotheVMwareSD-WANGatewayand learnstheprefixesbelongingtothetraditionaldatacenter.Asaresult,theVMwareSD-WANEdgeisabletoconnecttothetraditionaldatacenter.

Figure4.16:ExampleofVMwareSD-WANEdgetoaNon-VeloCloudSite(NSV).ConnectionisviatheVMwareSD-WANGateway.

Comparedwithhaving theVMwareSD-WANEdgedirectly formastandard IPSectunneltothetraditionaldatacenter,theapproachofutilizingtheVMwareSD-WANGatewayismuchmorescalableasitdramaticallyreducesthenumberoftunnelsneededfromindividualbranches.InordertoreachthescreenseeninFigure4.17,navigateto Configure > Profiles > {Profile Name} > Device Tab >Cloud VPN > Branch to Non-VeloCloud Site > Enable A configurationexampleforNVScanbeseeninFigure4.17.

18

Figure4.17:ConfigureCloudVPNforNVS

KeyTakeaways:

• Bydefault,allVMwareSD-WANEdges inaVMwareSD-WANsolutionwillbuildVCMPtunnelstoadesignatedprimaryandsecondaryVMwareSD-WANGateway.

• Inaddition,theSD-WANoverlaycanbeconfiguredtobuildbranch-to-branchVCMPtunnelsandadditionalVMwareSD-WANEdgetoVMwareSD-WANGatewayVCMPtunnelsforaccesstoNon-VeloCloudSites.

19

DeploymentModels

Overview

TheVMwareSD-WANsolutionenablesorganizations to connect tocloud-hostedapplications using an agile, transport-independent overlay that ensures privatenetwork performance, reliability, and manageability. There are differentapproaches that can be taken for the hosting of the solution itself, which aredetailedinthefollowing.

EnterpriseDeploymentModel(VMwareCloudHosted)

VMwareEnterpriseTopology(akaOTT:OverTheTop)makesuseofVMwareSD-WANGatewaysandVMwareSD-WANOrchestratorsco-locatedatmajor facilitiesaround the world to help enable the cloud-delivered strategy. This strategyleveragesproximityoftheVMwareSD-WANGatewaystoXaaShostinglocationstoremediatelast-mileconnectivityissuescommonlyseenatthebranch.Thisenablessupportof theshift tocloud-basedworkloadsandapplications inplaceofdoingtraditionalbackhaultothedatacenter.

20

Figure4.18:VMwareSD-WANOverTheToparchitecturewithcloud-hostedgatewayandorchestrator

TheVMwareSD-WANOrchestratorsarealsohostedincloudlocationsacrosstheworld, providing ready access. To further insulate from extended or significantoutagesinanyoneregion,eachVMwareSD-WANOrchestratorisprotectedwithadisasterrecoverypeerhostedinadifferentgeographicregion.AstheVMwareSD-WAN Orchestrator only participates in the management plane, any outageexperiencedherewouldhavenocontrolordataplaneimpactonVMwareSD-WANGatewaysorVMwareSD-WANEdgesconnectedtoit.Visibilityandmanageabilityofthese devices could be impacted, however, and so a redundancy strategy is inplacetomitigateanypotentialimpact.

The VMware SD-WAN Gateways in the cloud-hosted model are assignedautomatically via geolocation, with a primary and geo-redundant secondarygatewayallocatedperVMwareSD-WANEdges.InorganizationswherecloudVPNis enabled, a super gateway and redundant super gateway are also elected toensure that control plane resiliency is maintained. A super gateway is used asgatewayof last resort forVPNexchangepoint betweenbranch sites indifferentregionsasdepictedinFigure4.19.ThissupergatewayisassignedbytheVMwareSD-WANOrchestratorattheapproximategeographiccenteroftheorganization.

21

Figure4.19:VMwareSD-WANsupergateway

ServiceProviderDeploymentModel(Provider-hostedorOn-premisesDeployment)

Theserviceprovider-hosted topology isverysimilar to theVMwarecloud-hostedtopologyas itconsistsofthesamebasiccomponents. InthismodeltheVMwareSD-WAN Orchestrator and VMware SD-WAN Gateway will be located at serviceprovider-ownedandoperatedfacilities.Additionalaspectsoftheserviceproviderdeployment model such as partner gateways, are covered in a later section,Chapter8:ServiceProvider.

Figure4.20:VMwareSD-WANarchitecturefortheserviceprovider-hostedoffering

22

One of the main advantages in the service provider-hosted model is in thefunction of the gateway itself. This partner gateway functionality provides anadvantagetotheserviceproviderinthattheyarehostingthegatewaythemselves,providinglast-mileremediationbenefitstotheirvariousserviceofferings.

On-premises

Theon-premiseshostedtopologydiffersinthattheVMwareSD-WANGatewayandVMwareSD-WANOrchestratorarenolongercloud-hostedcomponents. Inthesedeployments, theVMwareSD-WANGatewaywill eitheractasa controllerorwillcarrybothcontrolanddataplanetraffic.Inon-premisesscenarios,wherepartnergatewaymodeisutilized,theconfigurationandtopologies,evenusecases,wouldcloselymirrorthosefoundinaserviceprovider-hostedtopology.

Figure4.21:VMwareSD-WANdeploymentoptions

23

KeyTakeaways:

• VMware SD-WAN has a nice simplified deployment model fororganizationswhereby they only have to install the VMware SD-WANEdgeandgetthecontrolplaneandmanagementplanefromtheVMwarehostedOrchestratorandGateway.

• Serviceproviderscandeploy theirownVMwareSD-WANOrchestratorandVMwareSD-WANGatewayformultiplecustomersandoptimizethecustomertraffic.ThisgivesSD-WANbenefitsforthelast-mileandaccesstoprivatenetworksforthemid-mile.

• Organization-hosted or on-premises deployment requires that theorganization or service provider install the VMware SD-WAN Edges,OrchestratorandGateways in theirownbranchesanddatacentersandoperatemanagementplaneandcontrolplanedirectly.

24

SiteTopologyandRedundancy

This chapter covers the commonly deployed topologies for VMware SD-WAN. Itdescribes the network topology, deployment considerations, and best practiceswhen inserting VMware SD-WAN Edge into a site. Figure 4.22 is a high-leveldiagramofthenetworktopologyshowingtheindividualsitetopologiescoveredinthis chapter. It shows a single data center and 5 branch sites. There are twounderlay transports shown—one MPLS and one internet service provider. TheVMware SD-WAN Orchestrators and Gateways are VMware cloud-hosted andreachablebyinternettransport.TheVMwareSD-WANEdgeswillattempttoreachthe VMware SD-WAN Orchestrators and Gateways and establish control planeconnectionsovereachtransport.

25

Figure4.22:TopologyandHighAvailabilityoverview

VMwareSD-WANEdgeRedundancyOptions

Therearethreedifferentdesignoptionsforredundancydesigns:

• HighAvailability(usedinbranchand/ordatacenter)

• Clustering(usedindatacenteronly)

• VRRP(usedinbranchonly)

Figure 4.23 shows the High Availability options in the UI,which can bereached by Configure > Edges > {Edge} > Device tab > High

26

Availability . The first two options will be discussed in the data centercontext.

Figure4.23HighAvailabilityoptions

DataCenterDesignOptions

Fordata center locations, there are two designs that are commonly used. Theyare:

• HighAvailability(HA)mode

• Clusteringmode

Therearedistinctcharacteristicsthatareassociatedwitheachoption.AsummarycomparisonbetweenthetwomodesisshowninTable4.1.

27

HighAvailabilityMode ClusteringMode

Highavailabilityforsiteswithasinglepairof

VMwareSD-WANEdges.Branchesorsmall

scaledatacenters.

Recommendedfordatacenterdesigns,

typicallynotapplicabletobranchdesigns.

LimitedtoonepairofVMwareSD-WAN

Edges.

AllowhorizontalscalingofVMwareSD-WAN

Edgestomeetperformancerequirements.

AddingVMwareSD-WANEdgesinaclusteris

simpleandnotdisruptive.

LANsideconnectiontodownstreamdevices

canleveragestaticroutesordynamicrouting

protocols(eBGP).

LANsideconnectiontodownstreamdevices

requiresdynamicroutingprotocol,likeeBGP

Capacityislimitedtotheactivedevice.CapacityisaggregatedacrossalltheVMware

SD-WANEdgesinthecluster.

Table4.1:HighAvailability/Clustercomparison

Eachdesignscenarioisdiscussedinmoredetailinthefollowingsection.

HighAvailabilityModeVMwareSD-WANEdgesconfiguredwithHAmodeendupwithexactly thesameconfiguration.OntheVMwareSD-WANOrchestrator,theywillshowupasasingleVMwareSD-WANEdgedevice.TherearetwooptionsforconfiguringHAmode,andthe VMware SD-WAN Edge will automatically select either option depending onhowtheWANinterfacesareconnected.ForminganHApairrequiresbothdevicestobeofthesametype.

Option1-StandardHAMode

Figure4.24providesanoverviewofStandardHAMode.Inthisdesign,theVMwareSD-WANEdgesconnecttodifferentswitchesontheWANandLANinterfaces.ThetwoVMwareSD-WANEdgesmusthavephysicalmirror connectionsonboth theWANandLANinterfaces.TheVMwareSD-WANEdges,oneactiveandonestandby,are physically connected back-to-back to establish a failover link. The standbyVMwareSD-WANEdgeblocksallportsexcepttheHAportforthefailoverlink.HAlink is used for sending keepalive and communication of active / standby statebetweenthetwoedges.

28

Figure4.24:StandardHAmode

DeployingHighAvailability(HA)allowsforhitlessupgrades.

Full redundancy is achieved by connecting the VMware SD-WAN Edges toredundant LAN switches. Typically, they are L3 switches in HSRP/VRRPconfiguration. In order to achieve end-system connectivity on the LAN side, theVMware SD-WAN Edge pair needs to have a static route pointing towards theVRPP/HSRP Virtual IP address configured on the switches, or a dynamic routingprotocol(eitherBGPorOSPF).

OntheWANside,theremustbeeitheraWANmodemwithtwoportstoconnectbothVMwareSD-WANEdgesintheHApair,ortheremustbeanadditionalpairofswitchesprovidingtheWANconnectivitytobothVMwareSD-WANEdgesintheHApair.Forexample,Figure4.24showsahybridsitewithanMPLSandan internetconnection, which are provided to both VMware SD-WAN Edges through theseswitches.

The failoverfromactivetostandbyhappenswhentheedgesdetectthatphysicalconnectivityhasbeenlostoneithertheWANortheLANconnections.

Option2-EnhancedHAMode

Figure4.25showsEnhancedHAMode.Iteliminatestheneedforlayer2switchesontheWANside.Toenablethisoption,theWANconnectionsmustterminateondifferent interfaceportnumbers (see GE2 and GE3 on figure 4.25).When the

29

active VMware SD-WAN Edge detects different WAN link(s) connected to thestandbyedgewhencomparedtothelink(s)connectedtoitself,itwillautomaticallyselectenhancedHAmode.

Figure4.25:EnhancedHAMode

FortheactiveVMwareSD-WANEdgeto leveragetheWANlinkconnectingto thestandbyVMwareSD-WANEdge, theactiveVMwareSD-WANEdgeestablishesanSD-WANOverlaythrough theHA link.Traffic fromtheLANside is forwarded totheactiveVMwareSD-WANEdge,andthebusinesspolicydefinedwilldeterminehow the traffic flows are distributed across the links on both VMware SD-WANEdges.

In the event of a failure of the heartbeat link between the active and standbyVMwareSD-WANEdges, split brainprotectionwouldoccur automatically, as theVMwareSD-WANGatewayactsasawitness.Inthisscenario,theVMwareSD-WANEdgewhich used to be standby, and is now active, would build a tunnel to thesame VMware SD-WANGateway to which the currently active VMware SD-WANEdge in the pair has already established tunnels. When the VMware SD-WANGatewaycomparestheserialnumberofbothVMwareSD-WANedges,itwillnoticethe 'new'activeVMware SD-WANEdge, and it will instruct the VMwareSD-WANEdgewhichusedtobeactivebeforethefailovereventtogointostandbymode.

AllHighAvailabilitymodesareapplicabletobranchsitedeployments.

30

ClusterModeClustermodeallowsthedeployedVMwareSD-WANEdgesinthedatacentertobeused inaresource-awareclustersolution. Itcompletesthecommonredundancyin the data center switch fabric with an equally redundant SD-WAN clustersolution. Each new connection from a VMware SD-WAN Edge is dynamicallyassignedbytheVMwareSD-WANGatewaytotheleastbusyVMwareSD-WANEdgeinthecluster.AnoverviewtopologyisshowninFigure4.26.

Figure4.26:HubClusterdesign

In this mode, the VMware SD-WAN Edges are configured to join a hub clusterconfiguration.Alledgesinthehubclusterarepresentedasonehubtothebranchedges.TheVMwareSD-WANGatewaydeterminestheedge in thehubcluster towhichabranchedgewillbuildanoverlay tunnel.TheVMwareSD-WANGatewayselectsoneoftheedgesinthehubcluster,basedonlowestutilization,andassignsittoabranchedge.ThisisshowninFigure4.27.

EveryhubinthehubclusterreportsusageandloadstatisticstotheVMwareSD-WANGatewayperiodically.TheVMwareSD-WANGatewaymaintainsalistofhubssortedbyload.ThebranchVMwareSD-WANEdgerequeststheVMwareSD-WANGateway for thehubIPaddress.TheVMwareSD-WANGatewayassignsthe leastloadedhubtothebranchVMwareSD-WANEdge.Nostateissyncedbetweenthehubs in the cluster. The tunnel is set up by the VMware SD-WAN Edge to theassignedHub.

31

1

2

3

4

5

HerearethestepsarticulatedinFigure4.27.

Each hub reports usage and loads stats to the NSX SD-WAN Controller(VCC)periodically. VCCmaintains a list of hubs in an increasing orderoftheirload.

BranchVCErequestsVCCforhubIPaddress

VCCassignsleastsloadedhubtothebranchVCE

Therewillbenostatesyncbetweenthehubsinthecluster

BranchVCEsetsuptunneltotheassignedhub.

Figure4.27:Resource-awarehubclustering

There is aconfigurationoption in theVMwareSD-WANOrchestrator thatwouldallow the VMware SD-WANGateway to automatically rebalance the connectionsover time. The VMware SD-WAN Edges in the cluster do not communicatewithotherVMwareSD-WANEdgesinthesamecluster.

Each cluster member will have its own IP addressing for the WAN and LANinterfaces.AlltheVMwareSD-WANEdgesinthehubclusterarerequiredtoruna

32

dynamicroutingprotocol,e.g.eBGP,withthelayer3device(s)ontheLANsidewithauniqueAutonomousSystemNumber(ASN)foreachclustermember.ThisallowstheVMwareSD-WANEdges in thehubcluster toadvertise theLANprefixes intotheoverlaytunnelsanddrawtrafficintothedatacenter.WhenaVMwareSD-WANEdge in thehubcluster loses itsroutingadjacencyontheLANside, theVMwareSD-WANOrchestratorremovestheVMwareSD-WANEdgeinquestionfromthelistof available hubs and re-balances the VCMP tunnels on that device to theremainingclustermembers.

Hubclustering isused forhorizontalscaleandredundancy forVMwareSD-WANEdgesinthesamedatacenterlocation.HubclusteringisrecommendedoverHAfordatacenterdeploymentinmostcases.AtleastN+1hubsarerecommendedina cluster, assuming N is the minimum number of hubs needed to meet thebandwidthandtunnelrequirements.

BranchDetails

ThereareseveralbranchtopologiescommonlydeployedusingVMwareSD-WAN.Theyare:

• SingleVMwareSD-WANEdgewithinternet-onlyconnection.

• SingleVMwareSD-WANEdgewithMPLSandinternetconnections.

• DualVMwareSD-WANEdgeswithMPLSandinternetconnections.

• SingleVMwareSD-WANEdgeandatraditionalrouterin-path.

• Single VMware SD-WAN Edge and a traditional router in a VRRPconfiguration.

ThefollowingwillcovertheVMwareSD-WANinsertiontopologies.

SingleVMwareSD-WANEdgewithinternet-onlyconnectionThe internet-only sites will build a SD-WAN Overlay to the hubs using internettransport. Ifdynamicbranch-to-branchVPNisenabledintheprofileforthesites,the sites will build on-demand overlay tunnels to other sites connecting to theinternettransport.ToreachsitesthatareonlyconnectedtoMPLStransport,thetraffic froman internet-only sitewill have togo through thedata center orhublocation.Figure4.28showsthistopology.

33

Figure4.28:Edgewithinternet-onlyconnection

SingleVMwareSD-WANEdgewithMPLSandinternetconnectionsSites using a single VMware SD-WAN Edge with both MPLS and internetconnectionswillbuildaSD-WANOverlaytothehubsusingbothMPLSandinternettransports. When the Hub is used as the transit location for branch-to-branchcommunication,theVMwareSD-WANEdgeatthebranchlocationsisnotrequiredtorundynamicroutingprotocolssuchasBGPwith theprovideredgerouterontheMPLS interface. The only requirement for the VMware SD-WAN Edge at thebranch location is to have adefault routeon theWAN interface that points thetraffictotheprovideredgerouter.Figure4.29showsthistopology.

Figure4.29:EdgewithMPLSandinternetconnection

DualVMwareSD-WANEdgeswithMPLSandinternetconnectionsSitesusingadualVMwareSD-WANEdgepairwithMPLSandinternetconnectionwillhavethesamedesignpossibilitiesasthehigh-availableexamplesdiscussedatthebeginningofthischapterunderDataCenterDesignOptions(standardHAmode,enhancedHAmode).Figure4.30showsthistopology.

34

Figure4.30:DualEdgewithMPLSandinternetconnection

SingleVMwareSD-WANEdgeandatraditionalrouterOff-PathForthesitesusingatraditionalcustomerequipment(CE)routertoterminateWANconnections, oneVMware SD-WANEdge canbe inserted and take advantageofdualcircuitscomingtoasite.TheVMwareSD-WANEdgewillrunadynamicroutingprotocolsuchasBGP,andexchangerouting informationwiththeCErouter.TheCErouterwilladvertisetheVMwareSD-WANEdgeinterfaceaddressintotheMPLSnetwork, thus providing reachability to the edge for other parts of the MPLSnetwork.TheVMwareSD-WANEdgecanthenformtheSD-WANOverlaysthroughbothtransportsconcurrently.Figure4.31showsthistopology.

Figure4.31:Edgeandtraditionalrouterin-path

SingleVMwareSD-WANEdgeandatraditionalrouterinaVRRPconfigurationThistopologyisthesameasthepreviousone,butnowVRRPisactivebetweenthetraditionalrouterandtheVMwareSD-WANEdge.TheVMwareSD-WANEdgeistheVRRPmaster and provides benefits from the SD-WAN Overlay functionalities innormalconditions.IncaseofaVMwareSD-WANEdgefailure,thetraditionalrouterisavailableasafallbackoption.

35

Thissetupiscommonformigrationscenariosandnotcommonlyusedinday-to-dayoperations.

KeyTakeaways:

• Atthedatacenter,thecustomerhasthechoicewithVMwareSD-WANto choose between High AvailabilityMode with up to two edges orClusterModewithuptoeightedgesinacluster.

• VMware SD-WAN brings a lot of flexibility to the branch. All variantsfromoneedgewithoneortwo linksuptotwoedgeswith internetandMPLS links in a redundantmode arepossible.Migration scenarios arefullysupported.

To learn more, visit:

www.velocloud.com

VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com. Copyright © 2019 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at

http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. and its subsidiaries in the United States and other jurisdictions. All other marks and names mentioned herein may be trademarks of

their respective companies. Item No: vmw-wp-temp-word 2/19