Post on 09-Feb-2022
Migration Strategy for 40G/100G Ethernet over40G/100G Ethernet over
Multimode FiberPaul KishBeldenBelden
OutlineOutline• The need for speedp• Evolution of network cabling• 40GBASE SR and 100GBASE SR standard• 40GBASE-SR and 100GBASE-SR standard• Relative economics – 10G vs. 40 G over MMF • Migration from 10G to 40G/100G Ethernet• Some example implementations• Optical loss testing accuracy• Road ahead for optical fiber• Road ahead for optical fiber
THE NEED FOR SPEED40G/100G Ethernet
THE NEED FOR SPEED
Global IP Traffic ForecastGlobal IP Traffic Forecast
Annual global IP traffic Major milestonesAnnual global IP traffic Major milestones
• By 2011 there will be more networked devices than people on earth
• By 2012 Internet video will surpass 50 % of consumer Internet traffic
• By 2015 the annual global IP traffic will reach the zettabyte threshold (966 exabytes)
CAGR 2010 – 2015 = 32%
1 EB = 1 000 000 000 000 000 000
Source: Cisco Visual Networking Index
1 EB = 1,000,000,000,000,000,000
Putting it into PerspectivePutting it into Perspective
byte gigabyte terabyte petabyte exabyte zettabyte8b 000 000 000 000 G 000 000 000
Time to transfer information @ 10 Gbps
1 B = 8b 1,000,000,000 B 1000 GB 1000 TB 1000 PB 1000 EBseconds 1minutes 15
days 10days 10years 30
centuries 300
The amount of information is finite
The scale of the information is staggering
Network Drivers for Higher Speed Ethernet
Multi-core Servers - high-performance compute (HPC) clusters where many servers are networked together to execute large y g gcomputing tasks through parallel processing
Virtualization Trend - consolidating workloads onto fewer, more highly utilized servers using virtualization technologieshighly utilized servers using virtualization technologies
Networked Storage Trend - networked storage shared by multiple servers is replacing the local hard disks in traditional servers
I/O Convergence Trend – servers are beginning to utilize a single converged Ethernet I/O interface to connect to separate storage area networks and local area networks
Data Center Network Aggregation Trend – as the deployment of 10GbE increases, there is a need for higher speed switch uplinks for network aggregation in the data center
* Ethernet Alliance white paper: “Overview of Requirements and Applicationsfor 40 Gigabit and 100 Gigabit Ethernet”
Networking Data RatesNetworking Data Rates40 GbE and 100 GbE Networkingg
EVOLUTION OF NETWORK CABLINGEVOLUTION OF NETWORK CABLING
The Evolution of Multimode Optical Fiber Network Cabling
2005
2010 LO 50 MMF (OM4)
10GBASE SR
40/100GBASE-SR
1995
2000LO 50 MMF (OM3)
50 MMF (OM2)
10GBASE-SR
1000BASE-SX
1990 62.5 MMF (OM1)FDDI
Ethernet Over Multimode Optical Fiber Cabling
SpeedMMF TypeDistance (m)
Name Standard DescriptionDistance (m)
1 GbpsOM1: 275OM2: 550 1000BASE-SX
IEEE 802.3z1998
850 nmVCSEL
2OM3: 8001998 2 MMFs
10 Gbps
OM1: 33OM2: 82
10GBASE-SRIEEE 802.3ae
850 nmVCSEL
10 GbpsOM3: 300OM4: 450
10GBASE SR2002 2 MMFs
40/100 OM3: 100 40GBASE-SR4 IEEE 802.3ba850 nm
40/100 Gbps
OM3: 100OM4: 150
40GBASE SR4100GBASE-SR10
IEEE 802.3ba2010
VCSEL8/20 MMFs
40G and 100G ETHERNET STANDARD40G and 100G ETHERNET STANDARD
40G and 100G Ethernet StandardIEEE Std 802.3ba-2010 standard was approved at the June 2010 IEEE Standards Board meeting. The standard includes:
• Copper40GBASE-CR4: 40 Gb/s over twin-ax copper cabling up to 7 m100GBASE CR10 100 Gb/ i bli 7100GBASE-CR10: 100 Gb/s over twin-ax copper cabling up to 7 m
• Multi mode fiber40GBASE SR4: 40 Gb/s over multi mode fiber up to 150 m40GBASE-SR4: 40 Gb/s over multi mode fiber up to 150 m
100GBASE-SR10: 100 Gb/s over multi mode fiber up to 150 m
• Single mode fiber /40GBASE-LR4: 40 Gb/s over single mode fiber up to 10 km100GBASE-LR4: 100 Gb/s over long single mode fiber up to 10 km100GBASE-ER4: 100 Gb/s over single mode fiber up to 40 km
40GBASE-SR4 Parallel Optics40GBASE SR4 Parallel Optics10GBASE-SR
Tx Rx10 Gb/s
Tx12
1 lane (2 MMF 850 nm)10 Gb/s
Rx
Rx
Tx
40GBASE-SR410 Gb/s
Tx12
Rx
234567R T
10 Gb/s•••
10 Gb/s
789101112
Rx Tx
10 Gb/s•••
13
124 lanes (8 MMF 850 nm)
100GBASE-SR10 Parallel Optics100GBASE SR10 Parallel Optics100GBASE-SR10
T R10 Gb/s
Tx123456
Rx
••
••
10 Gb/s
6789101112
Tx Rx• •
10 Gb/sRx 1
2345
Tx
• ••
10 Gb/s
6789101112
Rx Tx
••
••
14
10 lanes (20 MMF 850 nm)
IEEE 802.3ba Medium Dependent Interface (MDI)40G & 100 G Ethernet Over Multimode Fiber40G & 100 G Ethernet Over Multimode Fiber
Clause 86.10.3.3 Medium Dependent Interface (MDI) requirementsThe MDI adapter shall meet the dimensional specifications of IEC 61754-7p pInterface 7-3, the MPO adapter interface. … The MDI shall optically matewith the plug on the optical fiber cabling.
40GBASE-SR4 & 100GBASE-SR10 Using MPO Interface
40GBASE-SR4 Optical lane assignments
-100GBASE-SR10 Optical lane assignments100GBASE SR10 Optical lane assignments
40GBASE-SR4 Channel Specifications
PMD C i PMDC i
Fiber Optic ChannelMDI MDI
4 lanes
Description Type OM3 OM4 Unit
PMD Connection PMDConnectionPatchCord
PatchCord
Cable
Description Type OM3 OM4 Unit
Operating Distance Max 100 150 m
Cabling Skew Max 79 ns
Cabling Skew Variation Max 2.5 ns
Channel Insertion Loss Min 0 dB
Channel Insertion Loss Max 1.9 1.5 dB
The total connection loss allocated for OM4 MMF is 1 dB maximum (or 0.5 dB per connector)
17
RELATIVE ECONOMICSRELATIVE ECONOMICS
Equipment Pricing ScenarioEquipment Pricing Scenario• Some equipment examples:
Layer 3 Switch - 52 Slot (48 x 10G + 4 x 40G)Switching Throughput: 1.28 Tbps
QSFP+40GBASE-SR
Module
SFP+10GBASE-SR
Module
10G SRServer Adapter
2 ports
Relative Economics
Equipment (both ends) $/port/10GbE
Relative Economics
$3,500.00
Equipment (both ends) $/port/10GbE
$2,000.00$2,500.00$3,000.00
$500.00$1,000.00$1,500.00
$0.00$500.00
MMF 10G Sw-Sw MMF 10G Sw-Svr MMF 40G Sw-Sw
Vendor A Vendor BVendor A Vendor B
MIGRATION FROM 10G TOMIGRATION FROM 10G TO 40G/100G ETHERNET
Pre-terminated CablingPre terminated Cabling
• The use of pre-terminated optical fiber cabling canThe use of pre terminated optical fiber cabling can facilitate the migration path from 10G Ethernet to 40G and 100G Ethernet for– Switch-to-Server connectivity– Switch-to-Switch connectivity
Migration from 10GBASE-SR to 40GBASE-SR4
2 x 12-fiber MPO connectors
Cabling for 10GBASE-SR
1 x 24-fiber trunk cable assemblyMPO connectors
2 MPO t2 x MPO to12 x duplex LC
Cassette
Cabling for 40GBASE-SR42 x 12-fiber MPO connectors
6 x 24-fiber trunk cable assemblies
12 x MPO adapter
Optical Loss Budget Calculations
100 meters
Using OM3 or OM4 fiber cables• MPO connectors as specified in TIA 568-C.3 have a loss of up to 0.75 dB per connector allows a maximum of two
0.75 0.75
100 metersconnector allows a maximum of two connectors for a 100 meter channel
• Pre-terminated cable assemblies are il bl th t l l MPO
100 meters
available that use low loss MPO connectors with a maximum loss of 0.35 dB per connector
• The use of low loss connectors allows
Worst case
0.35 0.35 0.35 0.35The use of low loss connectors allows up to four connectors for a 100 meter channel
100 m Link length Worst case connectors Low Loss connectors
No. of Connectors 2 2 3 4Connector Loss (dB) 1.5 0.7 1.05 1.4
Optical Fiber Loss (dB) 0.3 0.3 0.3 0.3Optical Fiber Loss (dB) 0.3 0.3 0.3 0.3Channel Insertion Loss (dB) 1.8 1.0 1.4 1.7
meets 1.9 dB budget
Optical Loss Budget Calculations
• The optical loss calculations for 150 h l h i h
Using OM4 fiber cablesand low loss connectors
meter channels are shown on right
• Up to three, low loss, MPO connectors can be accommodated for a 150 m h l i i d OM4
150 meters
channel using pre-terminated OM4 optical fiber cable assemblies
0.35 0.35 0.35
150 m Link length Low Loss connectors
No. of Connectors 2 3Connector Loss (dB) 0.7 1.05
Optical Fiber Loss (dB) 0.45 0.45Channel Insertion Loss (dB) 1.05 1.5
meets 1.5 dB budget
Optical Fiber Array PolarityOptical Fiber Array Polarity
• TIA 568-C.0 standard outlines sample methods ATIA 568 C.0 standard outlines sample methods A and B for maintaining the polarity of parallel array systems
• The following slides illustrate method A and Method B polarity schemes, as well as a version of Method B polarity scheme using Type A (key up to key down) MPO adapters
40G Optical Fiber Array PolarityMethod A for Parallel SignalsMethod A for Parallel Signals
40G Optical Fiber Array PolarityMethod B for Parallel SignalsMethod B for Parallel Signals
40G Optical Fiber Array PolarityMethod B Using Type A AdaptersMethod B Using Type A Adapters
EXAMPLE IMPLEMENTATIONSFiber Connectivity
EXAMPLE IMPLEMENTATIONS
Example ImplementationExample Implementation Migrating from High Density
LC Duplex ModulesMigrating from Standard Density
LC Duplex Modules LC Duplex Modulesto MPO Adapters
• 18 connectors per unit
LC Duplex Modulesto MPO Adapters
• 12 connectors per unit
10G 10G
40G 40G
50% Increase in Capacity50% Increase in Capacity
Example Implementationh h lHigh Density Patch Panel Design
4 cassettes / 48 LC Duplex ports 4 HD Adapter frames / 72 MPO ports
Example Implementationp p
High density rack mountU t 72 LC D l /MPO P t
1U
High density rack mount fiber connectivity
t
Up to 72 LC Duplex/MPO Ports
2U systemsUp to 144 LC Duplex/MPO Ports
2U
4U
Up to 288 LC Duplex/MPO Ports
MPO Trunk Cable AssembliesMPO Trunk Cable Assemblies
12 fib MPO T k bl bli d t i
Cable construction Type of Connectors
12-fiber MPO Trunk cable assemblies are made to measure using a Product Engineering Code that specifies
Flame ratingFiber typeFiber count
Polish / performancePulling eyeLength in meters
MPO Trunk Cable AssembliesMPO Trunk Cable Assemblies
MPO T k bl bli i lti l f 12 fib d t
Cable constructionFlame rating
Type of ConnectorsPolish / performance
Fan-out Length,Construction
MPO Trunk cable assemblies in multiples of 12 fibers are made to measureusing a Product Engineering Code that specifies
Flame ratingFiber typeFiber count
Polish / performancePulling eyeLength in meters
Construction,and Geometry
OPTICAL LOSS TESTINGIt’s not as simple as it seems
OPTICAL LOSS TESTING
Accurate Loss MeasurementsAccurate Loss Measurements
• Why is it important?Why is it important?– Tight loss budgets for a channel
Measurement uncertainty can quickly use up the– Measurement uncertainty can quickly use up the available the loss budget
Testing Accuracy
Several major factors can affect the testing f ti l l taccuracy for optical loss measurements
1. the type and quality of the “test reference cord”
2. fiber mismatch between the test reference cords and the link under testcords and the link under test
3. the characteristics of light source and how light i l d i h fibis coupled into the fiber
Type and Quality of the “Test Reference Cord”Type and Quality of the Test Reference Cord
• end face geometry can have a significant effect on the test• end face geometry can have a significant effect on the test results and must meet precise parameters such as radius of curvature, apex and fiber protrusion
• the performance can be greatly diminished when the surface of the cord connector is scratched or dirty
– inspect the fiber connector with a fiberscope and clean the connector if it is dirty with a dry cleaning technique
Are Your Optical Loss Measurements Reliable?Are Your Optical Loss Measurements Reliable?
• Connect a “known good patch cord” to your equipment as if it g p y q pwere a fiber link and test it using the same test reference cords that are normally used for link testing
• Compare the measured result in the field to the loss value of the factory tested patch cord
• If the result is significantly different (greater than 0.2 dB), something is wrong
– It can be your equipment, your test method or your test cords
Fiber Mismatch Loss
• Fiber mismatch loss is an intrinsic coupling loss due top g
Core diameter differences
numerical aperture differencesnumerical aperture differences
core and cladding concentricity
• Fiber mismatch loss can be quite significant when connecting two fibers at the minimum and maximum of tolerance limits
• Fortunately, most fibers today are manufactured to tighter tolerances than worst case specifications
Launch Conditions
Launch conditions can have the greatest effect on optical l t f lti d fib
• different distributions of launch
loss measurements of multimode fiber
power (launch conditions) can result in different loss measurements
th l h t i d l
launchspot size and
angular • the launch spot size and angular distribution is controlled through
an optical system that couples the light into the fiber
angulardistribution
What is Encircled Flux?What is Encircled Flux?• Encircled flux (EF) is a new method of controlling the power
distribution of light that is launched from the end of a testdistribution of light that is launched from the end of a test reference cord (TRC)
• EF was developed by optical experts to reduce variability in li k l t d t t diti l th d th tlink-loss measurements compared to traditional methods that use an overfilled launch (OFL) source and a mandrel wrap
• The Encircled Flux test method is specified in TIA TIA-526-14-pB, “Optical Power Loss Measurements of Installed Multimode Fiber Cable”
• Encircled Flux is also the test method that is specified in the• Encircled Flux is also the test method that is specified in the IEEE 802.3ba Standard for 40 Gb/s Ethernet (40GBASE-SR4) and 100 Gb/s Ethernet (100GBASE-SR10).
EF Template for 50 μm Fiber at 850 nm
Reference: TIA-526-14-B, October 2010 Source: TIA-526-14-B
What is the Difference in Optical Loss Measurements Using OFL versus EF ?Measurements Using OFL versus EF ?
• the optical loss of three different 20 meter, 2-connector, permanent links p , , pwas measured using two different OLTS instruments
• the same test reference cords (using regular MMF) and mandrels were used when testing the permanent links for both instrumentswhen testing the permanent links for both instruments
• the test setup is shown below:
LC LC
20 m
LC LC
Receive cordi – 22mm Mandrel a) PL MMF
b) PL BIMMF Vendor_Ac) PL BIMMF Vendor_B
OLTS #1
i 22mm Mandrelii - EF Module
OLTS #1OLTS #2
Insertion Loss Measurements
Insertion Loss on 20m Permanent Link with two LC connectors
0 200
-0.100
0.000
OLTS #1 TRC #1
22mm mandrel
OLTS #1 TRC # 2
22mm mandrel
OLTS #2 TRC #1
22mm mandrel
OLTS #2 TRC # 2
22mm mandrelOLTS #2 EF module #1
OLTS #2 EF module #2
OLTS #1 EF module #1
OLTS #1 EF module #2
-0.500
-0.400
-0.300
-0.200
-0.900
-0.800
-0.700
-0.600
IL (d
B)
-1.300
-1.200
-1.100
-1.000
OFL OFL EFL EFL
0.5 dB
-1.400
Test ConditionsVendor A BI-MMF (PL 20m) Regular MMF (PL 20m) Vendor B BI-MMF (PL 20m)
OFL OFL EFL EFL
Optical Loss Testing SummaryOptical Loss Testing Summary• Optical loss testing in the field is not as simple as it
seems and can be affected by many variables includingseems and can be affected by many variables, including type and quality of the test reference cords, fiber mismatch and launch conditions (OFL vs. EFL)
• The Encircled Flux test method is recommended by industry experts for accurate optical loss measurements for both regular MMF and BI MMFsfor both regular MMF and BI-MMFs
• Encircled Flux is also the test method specified for 40G and 100G Ethernet applicationsand 100G Ethernet applications
Something NewgNew Addendum to TIA 942-A “Cabling Guidelines for Data Center Fabrics”
Example of Data Center Fabric Fat-Tree ArchitectureAll access switches are connected to every interconnection switchRequires more switch-to-switch cabling than traditional architectureq g
The Road Ahead for FiberThe Road Ahead for Fiber
• There is a growing need for high speed switch-to-switch and g g g pswitch-to- server connections– Relative economics ($/port/10 Gbps) is favorable for
40GBASE SR d ith 10GBASE SR40GBASE-SR compared with 10GBASE-SR• The use of 12-fiber pre-terminated cabling with MPO
connectivity provides a seamless migration path from 10G to y p g p40G in the future
• Migration from 10G to 100G is also easily accomplished using 12 fib bl bli d d i ll i ltwo 12-fiber cable assemblies today and potentially using only
one 12-fiber cable assembly in the future (new 100G IEEE 802.3 study group)