DCI e SD-WAN - GARR
Transcript of DCI e SD-WAN - GARR
![Page 1: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/1.jpg)
DCI e SD-WAN StefanoZani(INFNCNAF)
S.Zani
WorkshopGARRRoma,29-31Maggio2018
![Page 2: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/2.jpg)
Data Center Interconnection (Casi d’uso) • DatacenterExtension• BusinessCon1nuityInteressespecificoperINFN:
LaquanItàdidaIcheverrannoprodoNdaLHCedovrannoessereanalizzaI,neiprossimiRUNsubiràunaumentoesponenziale(adinvesImentocostante).
!OccorreuIlizzaretuUelerisorsedisponibiliovunquesitrovino.
S.Zani
0.0#
50.0#
100.0#
150.0#
200.0#
250.0#
300.0#
350.0#
400.0#
450.0#
Run#1# Run#2# Run#3# Run#4#
CMS#
ATLAS#
ALICE#
LHCb#
2010201520182023PB
peryear
DCIrealizzaInegliulImianni:
• EstensionedelTIER1sualtricentridicalcoloscienIficogestendotuUelerisorsedicalcoloconlostessobatchsystemeconaccessoaimedesimidataset.
• CloudBursIng:Testsull’uIlizzodirisorsedicloudproviderpubbliciperassorbirepicchidirichiestadiCPU.
![Page 3: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/3.jpg)
Estensione TIER1 su nodi di ReCas Bari (dal 2016) Overlay (L3 VPN) con banda dedicata Casod’uso:21KHS06diRECASBARIadusodegliesperimenILHCcomeestensionedelTIER1(CNAFBologna)
40nodida64Core(AMDOpteron),256GBRAMBandateoricaidealeversoloStorage:Circa100Gb/s(Throughput5MB/spercore).BandadisponibileperilDCI:20Gb/s(L3VPNdelGARR)su2x10Gb/s(Dominidibroadcastsepara1)INodidiBarihannoIPdelCNAFesonoinstalla1eges11comeglialtriWNdelCNAF(stessobatchsystemdelTIER1).
EfficenzalegataalIpodiJob(CPUintensiveoI/OIntensive).JobCPUintensivegiranoconefficienzaugualeaquelliinterni,IjobpiùI/OIntensive,comeprevedibilehannounaefficienzaminore.
CNAF-NEXUS
131.154.152.0/2210.10.152.0/22
RX2.BA1CNAF-NEXUS RX2-BO1 Tier2-BAHuawei
131.154.128.0/17
PBR
GARRNETWORK
L3vpn-TIER1-Barivrf-target137:11
193.206.137.61/30193.204.219.110/30
600Km–9msRTT
CNAFT1
2X10Gb/s
BariRECAS
2X10Gb/s
Cache (21KHS06)
20Gb/sdisponibilisu100Gb/steorici!cache(300TBAFM)• SquidperCVMFS(FSperswdiesperimento)• FronIerSquidperAtlaseCMS(Datatransfercache)• DNSpermapparesquiddelT1suglisquidlocali
DATACENTEREXTENSION
![Page 4: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/4.jpg)
DCI fra CNAF ed LNL (L2 VPN per Business Continuity) Casod’uso:InfrastruUuradibusiness-conInuityperleapplicazioniITegesIonalidell’INFN.Requisi1direte:Linklivello2fraCNAFediLaboratoriNazionalidiLegnaroconRTT<5ms.
CNAF-NEXUS
GARRNETWORKL2vpn-CNAF-LNL
131.154.500/24
RX2-PD2CNAFSW RX2-BO1 LNLSW
131.154.50.0/24
120Km–2.1msRTT
CNAF
10Gb/s
LaboratoriNazionalidiLegnaro
VLAN1048VLAN1048
VmwareStretchedCluster
DBinsync(Dataguard)
DCIinpreproduzionerealizzatodaGARRuIlizzandounLSP(LabelSwitchedPath)MPLSedunaL2VPNfraiduerouterGARRpressolesediutente(inmododaridurrealminimolalatenza).SuquestolinkèpossibilepropagareleVLANnecessarieestendendoidominidibroadcastfraiduecentri.AncheinquestocasooltreallaconfigurazionedellaVPNinOverlayc’èanchel’uIlizzodounpathabandadedicata.GARRstaanchevalutandol’ipotesidirealizzarelaconnessionepuntoapuntodireUamentealivellotrasmissivo.
Sistemadisincronizzazionealivellodiblockdevice
DB DB
![Page 5: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/5.jpg)
DCI o DCE con NFV (Overlay)
Nel caso di uIlizzo di ingenI quanItà di risorse di calcolo su Cloud,Analogamenteacome“Noleggio”VMperfareelaborazioneda@,possorealizzare DCI u@lizzando NFV istanziando vRouter, vSwitch, VPNConcentratorovFirewalltramiteiqualirealizzarei“Tunnel”necessari.
S.Zani
DataCenter
Switch/Router GENERALIPNETWORK
VirtualSwitchVirtualRouter CloudProvider
L2/L3VPN
VXLAN
E-VPN
SoluzionidiquestoIpopossonoessereancheCompletamenteautomaIzzateconsistemidiorchestrazioneopenoproprietari:CrossVcenterNSX,Contrail,ACI,BigCloudFabric,IPInfusion,ecc.
Risorseremoteincloud
![Page 6: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/6.jpg)
Sperimentazionisucloudpubbliche!UIlizzodiTunnelfraleVMeglielemenIstreUamentenecessariadintegrareleVMnelBatchsystem.• Aruba• Unicredit• Clouditalia• Microsoh
PartecipazionealprogeioHNSciCloud:PreCommercialProcurement• Sperimentazione Cloud di IBM ! VPN IPsec LAN to LAN (RouterVyaUa(NFV))–(ANvitàterminata).
• T-SystemeREA(1200Corenel2018)!Dynfarm+peeringspecificisuvrf dedicato a banda riservata fra GEANT ed i Cloud ProvidercoinvolI.
S.Zani
VMOnCLOUD
Batch server
CE
INTERNET GRE
Conf Server
VPN
Configurazione
CE
BatchServer
VPNServer
Conf.Server
“Dynfarm”
TIER1CloudProvider
Test di Cloud Bursting su cloud pubbliche (Overlay su General IP)
![Page 7: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/7.jpg)
Data Center Interconnection (Ottico) ad alte prestazioni Casod’uso:AccordoINFN-CINECAperl’uIlizzodiunsoUoinsiemedellaparIzione(A1)delsupercalcolatoreMarconi(basatosu,Intel®Xeon®E5-2697v4)inphaseoutcomecalcolatoreHPCdiCINECAmamoltoefficaceperIworkflowIpicamenteHTCdelTIER1delCNAF.216Server(36physicalcores,256GBRAM)ThroughputteoricoversolostoragedelTIER1:>300Gb/sLadistanzafraCINECAeCNAFèdicirca17KmProblema:Garan1rel’I/Odiaccessoaida1cherisiedonosullostoragedelTIER1
DCIconInfineraCloudExpress2©(CX1200):• 1.2Tb/sin1U(12x100GbinterfacceEthernet)• Massimadistanza100-150Km• 5,5μslatenza• Finoa27Tb/ssuunafibramonomodale(stackingCX1200)• Altamenteconfigurabile:CLI,DNA(SistemasovwaredigesIonediInfinera)oin
configurazioneSDN(usandoleAPI)RESTCONF,gRPC
S.Zani
Linea
12x100GbEthernetQSFP28
![Page 8: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/8.jpg)
S.Zani
Nexus9516
Nexus9516
INFNT1
RisorseLocali232kHS06
CX1200
VPC
INFNCNAF
4x100Gb(LACPportchannel)
Usate5x100Gb(finoa12x100Gb)
n.1darkfiber(17Km)
NE10032
NE1072 NE1072NE1072NE1072NE1072
CX12005x40G
b
4x100Gb(LACPportchannel)
CINECA
216Server(36physicalcores,256GBRAM)~170kHS06,Intel®Xeon®CPUE5-2697v4
EstensionetrasparentedellaLANdelTIER1RTTasinLAN:(0.4ms)
CNAF- CINECA Data Center Interconnection In collaborazione con GARR
8
![Page 9: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/9.jpg)
DCI – Super channel. IltransponderuIlizzatoperlaconnessioneCNAF–CINECAapreunanuovastrada:siriesconoasfruUarelecaraUerisIcheIpichedegliapparaItrasmissividireUamenteconapparaIda“Datacenter”eliminandoicosIdicomplesseinfrastruUureperlagesIoneditrasmissivioroutercarrierclass.
S.Zani
.
.
.
System
In/O
ut
FMP-F-250
SC23SC17
SC1
SC22
SC4SC2
.
.
Mux-Demux
.
.
.
SystemIn/O
ut
FMP-F-250
SC23SC17
SC1
SC22
SC4SC2
.
.
EsistonoapparaI“Transponder”ingradodigesIrelestessecapacitàintegrandosiinunsistematrasmissivochegliconsentadicopriredistanzepiùampieegesIrecollegamenIversodesInazionidifferenI.
Mux-Demux
23SuperChannelBW:27.6Tb/s1LCSMFiber
CX1200
CX1200
CX1200
CX1200
…..
…..
100GEthernet
100GEthernet
DataCenterADataCenterBTransponder TransponderLANSwitch/Router LANSwitch/Router
![Page 10: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/10.jpg)
S.Zani
ROADMROADM
ROADM
Transponder
Transponder
Transponder
ROADM: (Reconfigurable Optical Add/Drop Multiplexer)
LANSwitch/Router LANSwitch/Router
LANSwitch/Router
100GEthernet
100GEthernet
100GEthernet
QuesIOADM“Programmabili”sonocomposIdaWSS(WavelengthSelecIveSwitch)checonsentonodigesIreinstradamentooNcoinmanieraflessibilee“Programmabile”.ModulidiamplificazioneoNcaconsentonodicopriredistanzepiùampie.
ManagementSystem
(MonoVendor)
![Page 11: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/11.jpg)
• CiclidivitadifferenIdeivarielemenI!possibilecambiareglielemenIintempidiversi• SoluzionimulIvendor!Maggioriperformacee/ominoricosIPerriuscireagesIreunsistemadisaggregatooccorreuncontrolplanecheriescaadinteragirecontuNglielemenI.UnasoluzioneSDNpossibilmentebasatasustandardaperIcosItuiscelabaseperoUenereprogrammabilitàedautomazionedellaretegeografica. S.Zani
Disaggregazione del sistema di trasporto ed Open Line System Ivendorchedispongonodisoluzionichevannodall’IPall’oNcohannodatemposviluppatocontrolplanemulIlivellomaiprovidercercanodiusciredal“Vendorlock”equindisicercadiandareversounmodellodisaggregato:
ROADMVENDORC
AmpliVENDORD
Modellototalmentedisaggregato
SDNController
TransponderVENDORA
TransponderVENDORA
ROADMVENDORC
AmpliVENDORD
FMP-F-250
Mux-Demux
VENDORB
FMP-F-250Mux-Demux
VENDORB
OpenLineSystem
![Page 12: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/12.jpg)
Varie tecnologie per SD-WAN e DCI con la parte ottica programmabile ROADM (Reconfigurable Optical Add/Drop Multiplexer) Open Line System
CoriantGrooveG30
3,2Tbs/sin1U
19,2Tb/on1Fiber
AdvaFSP3000CloudConnect3,6Tb/sin1U
34,4Tbson1fiber
InfineraXT3300“Meshponder”upto6000Km
1,2Tb/sin1U
27Tb/son1fiber
JuniperTCX1000ProgrammableROADM
25.6Tb/s
…………….
S.Zani
NONSOLOETHERNETAlcunitransponder(PaccheUo/ONci)Consentonoiltrasportodi:Ethernet,FibreChanneledInfiniband
![Page 13: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/13.jpg)
Open Optical Packet Transport white box (Materia di sperimentazione con GARR)
• VoyagerInambitoTIPsuiniziaIvadiFacebookènatoilprogeUoVoyagercherappresentailprimowhiteboxcheintegralefuzinalitàdiDWDM,“Transponder”ePacketSwitch/Router(OpenPacketDWDM).hUps://telecominfraproject.com/open-opIcal-packet-transport/
S.Zani
Voyager(OpenPacketDWDMWhiteBox)
12xQSFP28(100GE)–4X200GDWDM
BroadcomTomahawk,AC400(DSPASICOpIcs)FacebookFBOSS,CumulusLinux…
Partecipan1TIPACACIAADVACienaCiscoCoriant
FacebookInfineraJuniper
Lumentum
![Page 14: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/14.jpg)
DCI (Software Defined – WAN) (Sperimentazine in collaborazione con GARR)
S.Zani
NRENs(OpIcaltransportsystem)
DataCenter1
DataCenter2 DataCenter3
Transponder
API
API
L’introduzionediapparaIMisI(Packet/DWDM)programmabilipensaIperDCIaprelapossibilitàdiallocareVERICircuiIadalIssimacapacitàondemandenonsemplici“Tunnel”alivellidiastrazionepiùalta(chedifaUorichiedonounoverprovisioningdeilink)
SDNControllerOrchestrator
API
Transponder
Transponder
![Page 15: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/15.jpg)
Possibile usecase per DCI: il Data Lake
S.Zani
StorageDatacenter3
StorageDatacenter1 StorageDatacenter2DATALAKE
GliesperimenIHEPstannovalutandounmodellodidatamanagementcheprevedel’accessoapochimagrandi“DataLake”.
“IdatalakesonogruppidiStorageCentergeograficamentedistribuiIche,potenzialmenteuIlizzanodifferenItecnologiedistoragemasonogesIIedacceduIcomesingoleenItà”
![Page 16: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/16.jpg)
Possibile “Usecase” per SD-WAN : Accesso ai Data Lake
S.Zani
Ilmodellobasatosui“DataLake”dovrebbe:• RidurreilnumerodicopiedeidaIrispeUoaimodelliaUuali
• MinoricosIperstorageeminoricosIperlagesIone.• NecessitàdimaggioreconnekvitàWAN
.DATALAKE1
DATALAKE2
CompuIngCenter1
CompuIngCenter2
CompuIngCenter3
CompuIngCenter4
CompuIngCenter5
Linkadaltacapacitànx100Gb–Tb/s
![Page 17: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/17.jpg)
DCI ed SD-WAN: Conclusioni LetecnichediDCIstannodiventandosemprepiùcomunielereIdellaricercastannoevolvendoleproprieinfrastruUureperfornireserviziondemanddiconneNvitàadaltacapacità.
IlconceUodiretegeografica“Governata”viasovwarenonènuovoaglioperatoridirete,probabilmenteoggiperòèrealmentepossibileperl’utente,arrivarea“Programmare”lostratodiretechecosItuisceiltrasportoperleproprieapplicazioniuIlizzandointerfacceeprotocolliaperI.
C’èmoltodaimparare,moltodasperimentareemoltodafareassiemeaicolleghidelGARRedellealtrereIdellaricerca.
…Credosaràmoltodivertente.
S.Zani
![Page 18: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/18.jpg)
FINE
S.Zani
![Page 19: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/19.jpg)
Backup slides
S.Zani
![Page 20: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/20.jpg)
Alcune soluzioni overlay di DCI basate su SDN ed NFV
S.Zani
Soluzionidiestensionegeograficadeidatacenter,sonorealizzabilianchealivellodiambienIdivirtualizzazioneedorchestrazione:Alcuneesempidisoluzioni:CrossVcenterNSX,Contrail,ACI,BigCloudFabric,IPInfusion,ecc.
NSX:LatecnologiadioverlaysoUostantesonoleVXLAN.
OpenStack+ContrailNSX
ConContrailsiuIlizzanoE-VPNperl’estensioneL2
ContrailvRouter
![Page 21: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/21.jpg)
R&D su SDN e NFV in ambito HEP
S.Zani
E’statocreatoungruppodilavorosullavirtualizzazionedellareteinsenoadHEPiX(hUps://www.hepix.org)HEPIXNFVWG(Mailinglist:hUps://listserv.in2p3.fr/cgi-bin/wa?SUBED1=hepix-nfv-wg)IlgrupposiproponediesploraretecnologiediNFVeSDNchepossanoaiutareagesIreglienormiflussididaIprevisIdaiprossimiRUNdiLHCtenendopresentecheesperimentdialtrescienzecompeterannopesantementesullestesseinfrastruUuredirete.RaccoglielevarieesperienzefaUeinambitoWLGCmeiendoassiemeEsperimen1,Si1,NREN
MichiganUniversity:ProgeUoATLAS(AGLT2/Michigan,MWT2/ChicagoeKIT)consisteneldeploymentdiOVSall’internodiistanzediproduzionediStorage(dCacheinquestocaso)eprevededicollegaretuNgliOVSadununicocontrollerpergesIreunasortaditrafficshapingfraidiversistorageelementdeisiI.hUps://www.aglt2.org/wiki/bin/view/Main/Open_vSwitch/WebHome
![Page 22: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/22.jpg)
Diagram of Possible Future SDN Dev-Ops Testbed
S.Zani
SiteAAgentOVS
TransferNode(OVS+FDT/GridFTP)
TransferNode(OVS+FDT/GridFTP)
SiteBAgentOVS
PanDA/DaTriAgent
NSA_NNSA_1
STPA STPB
OVStail(sitedependent)
OVStail(sitedependent)
LHCONEp-t-pMul1-domain
Fabric
Request
circuit
Newpat
h
Starttransfer
1
DataPlane
ControlPlane
1)RequestWANcircuit2)IntegratecircuitwithOVS3)TransferonnewE2Epath2
IndevelopmentCurrentlyinplace
Interfaces
3
OriginalSlidefromRamiro/Azher,Caltech
![Page 23: DCI e SD-WAN - GARR](https://reader033.fdocuments.us/reader033/viewer/2022052004/6285ad5c6428b415ef488dc6/html5/thumbnails/23.jpg)
S.Zani
The suggested LHC computing model
Comp.
CERN
Internet
A limited number of powerful and efficient resource centres
Smaller centres or commercial providers
Store ... Comp.
DE