GÉANT fibre and lighting Designing the next generation GÉANT optical layer
description
Transcript of GÉANT fibre and lighting Designing the next generation GÉANT optical layer
connect • communicate • collaborate
GÉANT fibre and lightingDesigning the next generation GÉANT optical layer
Guy Roberts, DANTE13 September, 2010
connect • communicate • collaborate
Contents
GÉANT network PoPs, architecture, fibre types Wavelength services, regeneration and channel planning Growth projections
Fibre footprint Infrastructure analysis - route diversity Diversity case studies
40/100G 40G trial 100G and coherent, PM-QPSK and fibre types
GÉANT3 RFI: re-engineer the photonic layer? RFI process and outcomes
connect • communicate • collaborate
GÉANT Network
connect • communicate • collaborate
GÉANT network
Local campus networks link to NRENs which interconnect via GÉANT backbone
Transfer rates of up to 10Gbps across 50,000 km of network infrastructure
25 Points of Presence (PoPs), 44 routes and 18 dark fibre routes
connect • communicate • collaborate
Dark fibre topology
5
LON
PARVIE
COP
BRA
DUB
FRA
AMSBRU
GEN
MAD
BUD
MIL LJU
PRA
ZAGBAR
12,000 route km
Two dark fibre rings – Western Ring and Eastern Ring
Western Ring:UK-BE-NL-DE-CH-FR
Eastern Ring:SK-HU-HR-SI-AT
Rings interconnected: CH-IT-ATDE-CZ-SK
DF spurs/loops to:ES, IE, DK
Western Ring
EasternRing
connect • communicate • collaborate
A big European mesh...
GÉANT backbone is only part of the bigger picture
When NREN connectivity is overlaid can see full complexity
CBF – increasing role?
This is needs updating for: Rediris Garr Hungarnet
Updates please!
connect • communicate • collaborate
GÉANT Fibre
6 fibre providers: Level3 Colt Interoute TeliaSonera Invitel (previously
Memorex and Pantel) Global Crossing
G.655 (E-LEAF)• All Western ring• Madrid, vienna...
G.652 (SMF)• Routes to Copenhagen• Eastern ring (?)
below 200
200-399
400-599
600-799
800-999
1000-1199
1200-1399
1400-1599
1600-1799
1800-1999 over
2000
4
3
5 5
1 1 1
0 0 01
PoP-PoP distance (km)
below 50
50-59 60-69 70-79 80-89 90-99 100-109 110-120 over 120
14 1724
44
31
18
54 7
Hut-hut distance (km)
Some long routes!
connect • communicate • collaborate
Alcatel LightManager
Hybrid networking was introduced in GÉANT2
~100 x 10 Gb/s multi-hop wavelengths currently deployed
Lambda services delivered using Alcatel’s 1626LM equipment
1241km unregenerate reach on G.652 Frankfurt-Copenhagen
Attenuation up to 28dB between amplifiers
Point-to-point only – no ROADM functions used
40 Gb/s field trial successfully completed, these transponders now to be used for IP: Amsterdam-Frankfurt Frankfurt-Geneva
1626LM
connect • communicate • collaborate
Managed wavelength service
POP A
POP BPOP D
GÉANT
• 10G and 40G SDH clients• Static routing and OADM• Full rate 10GE LanPhy
POP C
Manually patched at intermediate sites
connect • communicate • collaborate
GÉANT 10G lambda traffic matrix(Feb 2010)
10
AMS FRA GEN PAR LON COP PRA MIL MAD LJU ZAG BRA BUD VIE BRU DUBAMS * FRA 4 * GEN 16 5 * PAR 2 2 4 * LON 4 5 4 4 * COP 3 4 2 0 2 * PRA 2 3 0 0 0 0 * MIL 0 2 5 0 0 0 0 * MAD 4 3 6 0 0 0 0 0 * LJU 0 0 0 0 0 0 0 0 0 * ZAG 0 0 0 0 0 0 0 0 0 1 * BRA 0 0 0 0 0 0 1 0 0 0 0 * BUD 0 0 0 0 0 0 1 0 0 2 2 1 * VIE 0 0 0 0 0 0 2 4 0 4 3 2 6 * BRU 2 0 0 0 2 0 0 0 0 0 0 0 0 0 * DUB 0 0 0 1 4 0 0 0 0 0 0 0 0 0 0 *
All end-to-end wavelengths: Wavelength services IP wavelengths GÉANT+ wavelengths
connect • communicate • collaborate
GÉANT: 100 x 10Gb/s wavelengths
All lambdas....
connect • communicate • collaborate
Wavelength exhaust?
System is designed for up to 40 wavelengths on all routes.
G.655 fibre requires spacing of 100GHz for first 20 wavelengths with 50GHz infill available for next 20 wavelengths.
Alcatel have modified their wavelength spacing rules over the past 4 years as they refine their modelling tools.
Currently fibre is approx. 50% full on Western Ring, 19 wavelengths on Amsterdam-Frankfurt and Frankfurt-Geneva spans.
Capacity planning projections suggest that design limit of 40 channels will be exhausted in lifetime of GÉANT3 project on some routes (if no 40/100G channels are deployed).
connect • communicate • collaborate
Dark fibre footprint
connect • communicate • collaborate
DANTE has an ongoing exercise to enter all GEANT dark fibre and leased wavelength routing into a Geographic Information System.
Currently using Google Earth – data is being distributed to NRENs
Fibre providers supply data of various formats and quality
Main goal is to identify shared risk groups (often very tricky to be certain about these; “intelligence” on deals between operators is often useful as well – swaps, purchases, etc)
Note that shared routes does not necessarily mean shared huts
For completeness this should (but does not yet) include NREN routes
Two case studies presented
Dark fibre routing
connect • communicate • collaborate
Fibre footprint, fibre types
fibre, routing
GÉANT dark fibre routes
connect • communicate • collaborate
4 dark fibre routes, 4 wavelengths and 1 CBF route all terminate in Frankfurt
DF routes go to: Amsterdam, Copenhagen, Prague, Geneva
Difficult to keep track of routing of shared routing risks. This could be a shared trench, shared duct or shared cable – these are not easy to distinguish
Frankfurt city authorities limit the number of roads available for fibres – increases risk of shared routes
Case study 1: Frankfurt metro
connect • communicate • collaborate
Case study 1 - Frankfurt metro
GÉANT: Frankfurt dark fibre metro routes
Shared risk?
connect • communicate • collaborate
Major fibre cut between Geneva and Basel - has happened at least once
Resulted in loss of both GEN-FRA and GEN-MIL
IP traffic between Geneva and Milan (and S&E of there) will reroute around a very long loop:
Geneva-Paris-London-Brussels-Amsterdam-Frankfurt-Prague-Bratislava-Vienna/Budapest...
If providing restorable wavelengths then substantial regen would be required to make the restoration path feasible
18
Case Study 2: LHC OPN and Geneva
connect • communicate • collaborate
To Paris
To Madrid
To Frankfurt
To Milan
Shared risk?
connect • communicate • collaborate
Case Study 2: LHC OPN and Geneva
LHCOPN procured 3rd party leased wavelengths
Analysis of LHCOPN performed by Michael Enrico (back in 2007) showed that leased wavelengths had some shared risks
For example, up to 8 wavelengths may share a routing risk between Geneva and Basel
Ways to improve resilience levels (3 examples, may be others): add a (GEANT) fibre route between Marseille (+ 3-d ROADM) and Milan add a third diverse fibre route out of Geneva to Milan - this will allow for
restorable wavelength services use a CBF route (ensuring it is based on diverse fibre) to introduce a 2nd
diverse lambda route between Geneva and Milan - this will allow for restorable sub-wavelength services and improve all round IP resilience
connect • communicate • collaborate
SRLG analysis
DEFrankfurt
Basel
T1 GRIDKA
T1
Zurich
CNAF
DK
Copenhagen
NL
SARA
UK
London
T1
BNL
T1FNAL
CH
NY
Starlight
MAN LAN
FR
Paris
T1
IN2P3
Barcelona
T1
PIC
ES
Madrid
T1
RAL
ITMilan
Lyon
Strasbourg/Kehl
GENEVA
AtlanticOcean
VSNL N
VSNL S
AC-2/Yellow
Stuttgart
T1 NDGF
T0
HamburgT1SURFnet(Between CERN and BASEL)
Following lambdas run in same fibre pair:CERN-GRIDKACERN-NDGFCERN-SARACERN-SURFnet-TRIUMF/ASGC (x2)USLHCNET NY (AC-2)Following lambdas run in same (sub-)duct/trench:(all above +)CERN-CNAFUSLHCNET NY (VSNL N) [supplier is COLT]Following lambda MAY run in same (sub-)duct/trench as all above:USLHCNET Chicago (VSNL S) [awaiting info from Qwest…]
T1
TRIUMF T1
ASGC
???
Via SMW-3 or 4 (?)
Case Study 2: LHC OPN and Geneva
connect • communicate • collaborate
40G and 100G
connect • communicate • collaborate
Alcatel 40 Gb/s p-DPSK
Technology/cost challenge led to gong delay between 10G and 40G: Alcatel & Lucent spent 10+ years investigating 40G solutions
Designed to have reach that is similar to existing 10G and no guard bands required
Partial DPSK uses filtering to fit into 50 GHz channel spacing – easy to mix with existing 10G wavelengths
Problems experienced by JANET with PMD are not seenso far on GÉANT fibres
Not clear if 40G has a future (even as an interim step) –is market moving more rapidly to 100G than envisaged sayeven 6 months ago?
P-DPSK
connect • communicate • collaborate
40G wavelength trial
CHIT
DE
40G SDH analyser
40G pass-through
40G
PIC
40G PIC
40G trial Frankfurt-Geneva-Milan
Juniper T1600 SDH STM-256 PICs
Ran stability tests with Xena 10G Ethernet testers & Monitored PM data
Stable operation observed
connect • communicate • collaborate
100 Gb/s - PDM-QPSK andcoherent detection
Alcatel has developed 100G using PDM-QPSK and coherent detection
Polarization Division Multiplexing – Quadrature Phase Shift Keying
Other vendors are also using similar approaches for 100G
Coherent detection (mixing with local oscillator at receiver) allows phase information to be retained, this enables digital signal processing (DSP) to compensate for chromatic dispersion.
Eliminates the need for dispersion compensation fibre modules (DCM).
Detection of polarization modes at the receiver also allows DSP to compensate for polarization mode dispersion (PMD).
Complex Forward Error Correction (FEC) mechanisms now able to provide up to 10dB or more receiver margin, but can add latency.
connect • communicate • collaborate
GÉANT3 RFI:Re-engineer the photonic layer?
connect • communicate • collaborate
GÉANT RFI process
GÉANT has been engaging equipment vendors in an RFI process
The goal is to understand technology options for GÉANT3
Particular focus on the WDM/OTN layer
Vendors have been provided with fibre characteristics in the Western ring of the GÉANT network
Respondents have been asked to provide designs based on fibre data and capacity assumptions described in ‘Reference Network’ – see next slide
connect • communicate • collaborate
Reference network for RFIApply to this reference network:
Full mesh of 3x10G (excl. Bru & Lux)Full mesh of 1x 40G (ditto)Full mesh of 1x100G (ditto)Combination of these
28
LON
BRU AMS
FRA
GENPAR
COP
LUX
G.655441km
G.655290km
G.6521122km
G.6521241km
G.655641km
G.655737km
G.655818km
G.655658km
connect • communicate • collaborate
All coherent transmission?
GÉANT network designed for 10G transponders, 100G coherent technology offers performance improvements
Should we re-engineer parts of the common photonic layer to take advantage of coherent technology? Addition of some new ILA sites (where huts have previously been “skipped”)
Removal of dispersion compensation fibre
The addition of gain equalisers in some ILA sites
This will require a significant up-front capital investment to replace 10G transponders with 100G muxponders – cost benefit analysis required.
RFI includes questions to help this process – decision also depends on other issues such as capacity growth projections and architecture choices.
connect • communicate • collaborate
Coherent transmission – RFI results
Easier for green field rollout
No discrete 10G coherent! Only 4/10x10G muxponders so 10G traffic matrix needs to be commensurate
How will this fit in with GÉANT fibre footprint?
30
Also used Raman!
0
5
10
15
20
25
30
35
DCM- mixed noDCM- mixed noDCM- SMFonly
Regen Reqs (vendor A)
40G coh
100G
0
5
10
15
20
25
Vendor A Vendor B Vendor B - ng100G
Regen Reqs (vendor A vs. B)(noDCM, mixed fibre)
40G coh
100G
connect • communicate • collaborate
ROADMS, to use or not to use?
Are ROADMs suitable to the GÉANT backbone?
Pros: Rapid service delivery Restoration using GMPLS Reduced regeneration cost on multi-hop routes
Cons: Complex and expensive Large floor space requirements Where 10x10G muxponders used all 10 sub-wavelengths have to be
demuxed (like old PDH) at add-drop sites Potential benefits of reduced regeneration is limited for GÉANT due to very
long routes Restoration requires a lot of extra spare regeneration capacity to support very
long restoration paths
connect • communicate • collaborate
Summary
It is important to share European fibre footprint information to reduce risk Common GIF system helpful for this process Role for CBF?
ROADMs much more appealing for GÉANT than in “pre-coherent era”
RFI results show a significant variation in reach between vendors Not sure why yet (better xponders?, optimistic/pessimistic and/or more/less
thorough modelling???)
The move to coherent technology – an opportunity exists to re-engineer the photonic layer - up-front investment needs cost-benefit analysis
All coherent transmission scheme certainly looks appealing but only if expected (lambda) traffic matrix is suitable long-haul lambdas (not just POP to neighbouring POP) and enough of them (e.g. 10G channels may come in chunks of 4)
32