10.14.20

© OFS Fitel, LLC 2020

White Paper:

MULTIMODE FIBER IN THE ENTRPRISE -

A STANDARDS UPDATE

StandardsEcosystemEnterprise network customers and vendors -in fact, the en7re data communica7ons industry - benefit from interoperable, mul7vendor industry standards developed by the interac7on and coordina7on between Telecommunica7ons Industry Associa7on (TIA) and Interna7onal Electrotechnical Commission (IEC) component/fiber standards, American Na7onal Standards Ins7tute (ANSI)/TIA and Interna7onal Organiza7on for Standardiza7on (ISO)/IEC structured cabling standards and Ins7tute of Electrical and Electronics Engineers (IEEE) Ethernet and InterNa7onal CommiHee for Informa7on Technology Standards (INCITS) Fibre Channel applica7on standards.

The ability to meet market demand for speed and transmission capacity standards cannot be overstated. For example, the growth of op7cal mul7mode 3 (OM3) fiber, now ubiquitous in enterprise networks, was both highly dependent on and enabled Ethernet standards like 10GBASE-SR and Fibre Channel standards like 4GFC and 8GFC. OM4 fiber’s demand growth is based on standards including 10GBASE-SR, 40GBASE-SR4, 100GBASE-SR4 and 16GFC/32GFC. More recently, this standards ecosystem has enabled the con7nued growth of OM4 fiber along with the introduc7on and growth of OM5 fiber, based on Ethernet standards like 50GBASE-SR, 100GBASE-SR2, 200GBASE-SR4, 400GBASE-SR8, 400GBASE-SR4.2 and Fibre Channel 64GFC. These Ethernet and Fibre Channel applica7on standards are supplemented by key Mul7-Source Agreements (MSAs) like 100G-BiDi, 400G-BiDi and 40G-SWDM4, 100G-SWDM4. Current Ethernet and Fibre Channel standards development projects based on 100 Gb/s electrical and op7cal signaling builds on and adds to that standards founda7on.

RecentTIA&IEC&andANSI/TIA&ISO/IECStandardsforMMFWithin the last few years, TIA & IEC and ANSI/TIA & ISO/IEC developed the OM5 specifica7on that extends the 850nm performance of OM4 out to 953nm to take advantage of Short Wavelength Division Mul7plexing (SWDM) and Bi-Direc7onal (BiDi) VCSEL technologies. Recently, IEC and TIA completed efforts to characterize OM3 and OM4 between 850 and 953 nm to further support mul7-wavelength applica7ons in brown-field installa7ons.

IEC SubcommiHee 86A, Fibres and Cables, published IEC 60793-2-10:2019 in May 2019. This document contains specifica7ons for all category A1 mul7mode fibers, with Annex E3 providing EMB guidance (informa7ve) for OM3/4 fibers at wavelengths between 850 and 953 nm (Fig. 1). This EMB guidance for OM3/4 was needed to support the work of the IEEE P802.3cm 400 Gb/s over Mul7mode Fiber Task Force that generated the IEEE Std 802.3cm™-2020 standard.

Figure 1: EMB guidance for OM3/4 MMF between 850 and 953 nm according to the IEC 60793-2-10 ed.7. TIA guidance for OM5 is shown for comparison.

TIA’s TR42 Telecommunica7ons Cabling Systems CommiHee and its TR42.12 subcommiHee Op7cal Fibers and Cables approved project authoriza7ons for the adap7on of IEC’s fiber specifica7ons; this includes changing IEC 60793-2 (General) to ANSI/TIA-4920000-C and changing IEC 60793-2-10 (MMF family) to ANSI/TIA-492AAAF. The laHer document replaced current TIA-492-AAAA, AAAB, AAAC, AAAD, and AAAE with a single document, adap7ng IEC 60793-2-10 to become TIA-492-AAAF, and was published in April 2020.

RecentFibreChannelandFCIAforMMFFC-PI-7,64GFC(serial)The most recently approved Fibre Channel physical layer standard is Fibre Channel - Physical Interface-7 (FC-PI-7), 64GFC (serial), which specifies 100 m reach for OM4/OM5 fiber and 70 m reach for OM3. FC-PI-7, 64GFC "Gen 7" has a nominal signaling rate of 28.9 GBaud with a PAM4 modula7on format, and has payload throughput per direc7on or data rate of 64 Gb/s. This standard was developed by Task Group T11.2 of Accredited Standards CommiHee of INCITS during 2016, 2017, and 2018. INCITS is accredited by ANSI and is affiliated with the Informa7on Technology Industry Council. INCITS 543-2019 (FC-PI-7) was approved on November 2019 and is available as an ANSI standard. 64 GFC solu7ons and products are now being shipped and deployed.

FC-PI-7P,256GFC(4x64GFC,parallel)Task Group T11.2 of Accredited Standards CommiHee INCITS recently developed Fibre Channel - Physical Interface-7P (FC-PI-7P), 256GFC (4 x 64GFC, parallel) standard during 2019 and 2020. FC-PI-7P is a 4-lane solu7on that meets the electrical and op7cal requirements per lane of FC-PI-7 and is specified for 100 m reach for OM4/OM5 fiber and 70 m reach for OM3. The standard is scheduled to be approved by the end of 2020 or early 2021.

FC-PI-8,128GFC(serial)The latest ac7ve Fibre Channel physical layer project proposal is Fibre Channel - Physical Interface-8 (FC-PI-8)/128GFC (serial), which was ini7ated in December 2017 and became an approved project in April

2018. The FCIA Marke7ng Requirements Document (MRD) for FC-PI-8 and the project proposal were reviewed in February 2018. The FC-PI-8 MRD has a target technical stability date of 2021 and product availability is targeted for 2022. The requirements include backward compa7bility to 32GFC/64GFC and 100 m reach for OM5/OM4. This project will leverage the work in IEEE P802.3ck 100 Gb/s, 200 Gb/s, and 400 Gb/s Electrical Interfaces Task Force for the electrical lane specifica7ons, the work in IEEE P802.3db 100 Gb/s, 200 Gb/s, and 400 Gb/s Short Reach Fiber Task Force for MMF op7cal link specifica7ons, and the work in OIF for 100 Gb/s op7cal lane work SMF op7cal link specifica7ons. Key technical challenges such as technical feasibility of 100 Gb/s VCSELs within targeted project comple7on dates, mee7ng backplane loss budget and handling host PCB loss will have to be solved, with progress made to date. A line rate of 112.2Gbps is possible (for comparison, line rate for Ethernet is 106.25Gbps likely). At February 2019 FC-PI-8 mee7ng, 128GFC MMF solu7ons – based on straw poll, there was unanimous support to consider a true serial 128GFC variant, a mul7-wavelength variant, 56Gb/s BiDi solu7on and a mul7-wavelength variant, 56Gb/s CoDi solu7on. Based on progress with 100 Gb/s per wavelength VCSELs since that 7me, true serial 128GFC variant will be the path forward. At the June 2019 La Jolla, CA, US FC-PI-8 mee7ng, Finisar contributed “100 Gb/s PAM4 VCSEL links feasibility, strawman link budget” [22], the first 100G/wavelength VCSEL technical contribu7on. This contribu7on used 940 nm, but at August 2020 mee7ng, wavelength close to 850 nm was recommended for alignment with IEEE P802.3db Task Force. At June 2020 mee7ng, it was decided to include receiver photo detector capable of 850 nm and 940 nm for 128GFC, so that future genera7ons of FC-PI could use 940 nm and s7ll be backward compa7ble to 128GFC. OM5’s wideband wavelength (850 nm – 953 nm) capabili7es provides future-proofing for Fibre Channel cabling infrastructure if Fibre Channel u7lizes longer wavelengths in later genera7ons of FC-PI.

RecentlyCompletedIEEE802.3forMMF802.3bsand802.3cdIEEE Std 802.3bs™-2017, 200 Gb/s and 400 Gb/s Ethernet, for 400GBASE-SR16, specifying 70 m reach for OM3 and 100 m reach for OM4/OM5 was completed in 2017.

IEEE Std 802.3cd™-2018, 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet, for 50GBASE-SR, 100GBASE-SR2 and 200GBASE-SR4, specifying 70 m reach for OM3 and 100 m reach for OM4/OM5 was completed in 2018.

802.3cm400Gb/soverMultimodeFiberIEEE P802.3cm 400 Gb/s over Mul7mode Task Force developed the IEEE Std 802.3cm™-2020 standard was completed January 2020. This Ethernet standard specifies 400GBASE-SR8 (70 m OM3, 100 m OM4/OM5) and 400GBASE-SR4.2 (70 m OM3, 100 m OM4, 150 m OM5).

The two 400 Gb/s MMF Physical Medium Dependent (PMDs) standardized are shown in Figure 2. 400GBASE-SR4.2 met the 4-pair objec7ve and uses 2 BiDi wavelengths per fiber. 400GBASE-SR8 met the 8-pair objec7ve. Both are 400GAUI-8 electrical and are 8x50G electrical to 8x50G op7cal.

Figure 2: 400G MMF standardized in IEEE 802.3cm

400GBASE-SR4.2 is the first IEEE 802.3 standard to use wavelength division mul7plexing (WDM) for MMF PMD and is a two-wavelength solu7on. It is the first Ethernet standards-based applica7on that will take advantage of OM5 fiber’s higher bandwidth at longer wavelengths. 400G-SR4.2 has reach of 70 m OM3, 100 m OM4 and 150 m OM5, and u7lizes a standard 12-fiber MPO connector interface, with 8 ac7ve fibers. There is an Op7cal Media Dependent Interface (MDI) lane assignment. 400G-SR4.2 provides a low-cost point-to-point link, while also allowing a 4x100G breakout, even though 100G-SR1.2 is not an IEEE standard and was not in scope of 802.3cm project. A 4x4 fiber shuffle allows a 32-port 400G switch to be used as a 128-port 100G switch (see Figure 3)

Figure 3: 400GBASE-SR4.2 point-to-point, breakout, and shuffle modes

400G-SR8 is an eight fiber-pair solu7on and has reach of 70 m on OM3 and 100 m using OM4/OM5. Both single row MPO-16 and 24f MPO (MPO-12 two-row) were chosen as MDI op7ons (see Figure 4) [1]

Figure 4: 400G-SR8 connector op[ons for MDI [1]

400GBASE-SR8 op7cs offer maximum flexibility for shuffle applica7ons. Figures 5 and 6 show point-to-point, 2x2 fiber shuffle, 4x4 fiber shuffle and 8x50G breakout capabili7es and applica7ons. The first deployment is 2x200G [2].

Figure 5: 400GBASE-SR8 point-to-point and shuffle modes

Figure 6: 400GBASE-SR8 breakout

Ethernet100Gb/sperwavelengthVCSELs“Lowercost,shortreach,opticalPHYsusing100Gb/swavelengths”CFIA Call-for-Interest (CFI) “Lower cost, short reach, op7cal PHYs using 100 Gb/s wavelengths” [3] was made in November 2019. The CFI had the following goal: “Matching the module I/O rate to the ASIC I/O rate reduces component count and complexity, leading to lower cost & power op7cal interfaces. SerDes rates on both switch ASICs and server network interface cards are expected to move to 100 Gb/s per lane in the next few years, driving current work in the IEEE P802.3ck and P802.3cu Task Forces. However, the highest speed, shortest reach PMDs over mul7mode fiber under development in Ethernet are built on 50 Gb/s op7cal lanes. As the electrical power & interconnect bandwidth of servers & accelerators are increasing, high-radix switches are also able to connect more servers, crea7ng a trend towards fewer servers-per-rack and thus longer server aHachment connec7ons at 100 Gb/s and higher. At the same 7me, the achievable passive copper cable reach with 100 Gb/s lanes is expected by some experts to be shorter than the reach with 50 Gb/s lanes. In order to develop the lowest cost and complexity op7cal modules for next-genera7on server aHachment, as well as shorter switch-to-switch connec7ons, with

emerging SerDes rates, we request a Call-for-Interest to assess support for a Study Group to develop the PAR and CSD for lower cost, short reach, op7cal PHYs using 100 Gb/s wavelengths.” [3]

Figure 7 [3] shows that SerDes, Server aHachment speeds, are increasing from 25GbE and 50GbE to 100GbE, which “drives the economics and evolu7on of switch fabrics and port counts.” [3]

Figure 7: Progress of SERDES speeds [3]

It is expected that Top of Rack (ToR) switches will give way to Middle of Row (MoR) switches as server counts in racks decrease, due to increasing power dissipa7on per server; and that server speeds will move to 100G (see Figure 8) [4], with reaches of up to 30 m being required.

Figure 8: Evolu[on from TOR switch architecture to MOR switch architecture [4]

At 100 Gb/s, the reach objec7ve of passive copper, direct aHach copper (DAC) is 2 m (from IEEE P802.3ck). While ac7ve op7cal cable (AOC) solu7ons can support 100 Gb/s and 30 m reaches, they require on-site installa7on, and are par7cularly difficult to manage when suppor7ng breakout, commonly used for switch to server connec7ons [3]. Pluggable transceivers with structured cabling, BASE-SR approach, has the following advantages: “server aHachment rates can be selected by grouping a number of SR8 ports together as required with structured cabling; supports server-row cabling objec7ves; enable pre-installed overhead cabling that supports mul7ple line rate genera7ons (50/100G); aHach to overhead cabling with short cords; repeat installa7on paHern for all server racks for installa7on efficiency; allow breakouts in structured cabling to support various server data rates (50/100/200G).” [3]. Figure 9 shows the advantages of a structured cabling approach [3].

Booth, Issenhuth, Microso_

Figure 9: Structured cabling approach to Fiber-to-the-Server [3]

The poten7al PMDs/use cases are: “100G-SR (SFP112); 400G-SR4 in quad module (400G QSFP56-DD ports) with breakout to 4×100GBASE-SR; Dual 400GBASE-SR4 in octal module (2×400G QSFP112-DD or OSFP112 ports) with breakout to 8×100GBASE-SR.” [3]

The Open Data Center CommiHee (ODCC) recently completed study of “Next genera7on Data Center Connec7ons in China,” which states: “MMF might s7ll have a cost advantage and is expected to evolve to 100G per wavelength” and “30m is a reasonable first goal for MMF links for AOC or transceivers in server interconnects” and “Longer distance MMF transceivers are needed for switch interconnects with transceivers”[3]. So, 50 m reach for switch to switch connec7ons may be required. Reaches of 70 m to 100 m are desirable for switch-switch connec7ons.

Another poten7al applica7on for 100 Gb/s signaling low-cost, short reach op7cal links is to address AI workloads, which are stressing interconnect speed and reach. “Demand on compute, memory, and interconnect bandwidth have each grown by ~10x year-over-year recently, in the AI sector. Ethernet is a good protocol for accelerator cluster interconnects. Longer reach and higher speed op7cal Ethernet links will support larger clusters with flaHer interconnects to support larger machine learning problems of the future” [3]. See Figure 10.

Figure 10: 100 Gb/s short reach op[cal link for AI accelerators [3]

In terms of technical feasibility, experts affiliated with Finisar/II-VI and Broadcom both showed data to support 100G/wavelength VCSELs. Expert affiliated with Finisar/II-VI presented “simula7ons from models based on early characteriza7on data show feasibility to ~50 meters; based on a well damped, well behaved VCSEL response, ~24 GHz, pre-emphasis (2-tap T-spaced FFE -0.5,1), 5-tap Rx FFE, 0.6 nm spectral width, 940 nm assumed for simula7ons.”

Expert affiliated with Broadcom presented that link simula7ons suggest feasibility of a 50m OM4 link for 50 GBaud VCSEL opera7ng at 850 nm with equaliza7on (pre-emphasis and Rx FFE) [24]. Separately, at China Interna7onal Optoelectronic Exposi7on (CIOE) 2019, Broadcom presented simula7ons for 855 nm, 50 GBaud/100 Gb/s, 25 GHz bandwidth VCSEL for 30 m – 50 m OM4 links, with sample availability in 1Q20 and product availability in 1Q21.

100 Gb/s per wavelength VCSELs will provide the next step in the evolu7on of and con7nued relevance of VCSEL-MMF links.

IEEE 802.3 Working Group voters approved the mo7on: Move that the IEEE 802.3 Ethernet Working Group authorizes the forma7on of a study group to develop a Project Authoriza7on Request (PAR) and Criteria for Standards Development (CSD) responses for "Lower cost, short reach, op7cal PHYs using 100 Gb/s wavelengths” with a vote of Yes: 78 No: 0 Abstain: 2

IEEE802.3100Gb/sWavelengthShortReachPHYsStudyGroupIEEE 802.3 100 Gb/s Wavelength Short Reach PHYs Study Group successfully developed and approved a PAR, CSD responses, and following objec7ves [5]:

1. Support a MAC data rate of 100 Gb/s, 200 Gb/s and 400 Gb/s

2. Support full-duplex opera7on only

3. Preserve the Ethernet frame format u7lizing the Ethernet MAC

4. Preserve minimum and maximum FrameSize of current IEEE 802.3 standard

5. Provide appropriate support for OTN

6. Support a BER of beHer than or equal to 10^-12 at the MAC/PLS service interface (or the frame loss ra7o equivalent) for 100 Gb/s opera7on

7. Support a BER of beHer than or equal to 10^-13 at the MAC/PLS service interface (or the frame loss ra7o equivalent) for 200 Gb/s and 400 Gb/s opera7on

8. Define a physical layer specifica7on that supports 100 Gb/s opera7on over 1 pair of MMF with lengths up to at least 50 m

9. Define a physical layer specifica7on that supports 200 Gb/s opera7on over 2 pairs of MMF with lengths up to at least 50 m

10. Define a physical layer specifica7on that supports 400 Gb/s opera7on over 4 pairs of MMF with lengths up to at least 50 m

The study group work was completed in May 2020.

IEEEP802.3db100Gb/s,200Gb/s,and400Gb/sShortReachFiberTaskForceIEEE P802.3db 100 Gb/s, 200 Gb/s, and 400 Gb/s Short Reach Fiber Task Force started mee7ng in June 2020.

The Task Force and Working Group established liaison communica7on from the IEEE 802.3 Working Group to INCITS T11.2 to foster exchange informa7on with FC-PI-8/128GFC group. Task Force also passed mo7on to adopt re-using exis7ng PCS/FEC/PMA.

Task Force has been addressing the following key items based on technical contribu7ons: technical/economic feasibility of reach longer than 50 m, one PMD or two PMD, MDI topics (angled MPO, very small form factor connectors, op7cal lane assignment for breakout applica7ons), Tx and Rx equaliza7on, bidirec7onal PMDs with two wavelengths, low power and/or low latency links, modal noise and modal-par77on noise power penal7es, re-use exis7ng PCS/FEC/PMA (mo7on passed), MM chroma7c dispersion, chip to module electrical and op7cal training (from FC-PI-8), fixed vs adap7ve pre-emphasis for 100G per lambda op7cal link, op7onal linear and non-re7med architecture for low cost/power.

There was a key contribu7on that showed encouraging evidence of the technical feasibility of exceeding the 50 m reach of the current adopted objec7ve by both simula7on and experimental studies. A 100 m OM4 is shown feasible with 9 taps of FFE in Rx equaliza7on and is suggested for a baseline proposal, enabling broad market poten7al for switch-to-switch applica7ons. Opera7on around 850 nm is recommended as the star7ng point [6]. See Figure 11.

Figure 11: Technical Feasibility of 100 m OM4 reach for 53.125 GBd PAM4 VCSEL [6]

Project goal is to update objec7ves, establish baseline and 7meline by November 2020. Target projec7on comple7on date is 2022.

SWDM4MSA,100G-BiDi&400G-BiDiMSAThe SWDM MSA developed and released specifica7ons for 40G SWDM4 and 100G SWDM4 on March 2017. Commercial 40GSWDM4 and 100G SWDM4 products were available by the end of 2017. The key value proposi7on of SWDM4, which u7lizes 4 wavelengths on 30nm spacing of 850nm, 880nm, 910nm and 940nm, is to u7lize much lower cost duplex fiber cabling and connec7vity instead of parallel fiber cabling and connec7vity.

100G BiDi uses 2 wavelengths, 857 & 908nm, to also extend the value of duplex cabling and connec7vity infrastructure. Commercial 100G BiDi products became available at the beginning of 2018.

The 400G BiDi MSA was announced in July 2018 to define “op7cal data link specifica7ons and promo7ng adop7on of interoperable 100 Gb/s and 400 Gb/s op7cal transceivers for 100 m link distance based on a dual wavelength bidirec7onal transmission technology in mul7-mode op7cal fiber (MMF).” The reach objec7ves are 70 m OM3, 100 m OM4 and 150 m OM5. Specifica7ons for 400G-BD4.2 were available September 2018.

The supported reaches for OM3, OM4 and OM5 for 100G-SWDM4, 100G-BiDi, 400G-BiDi, along with MMF PMDs from 802.3bs, 802.3cd and 802.3cm are shown in Table 2 below highligh7ng the extended reach benefit and future-proofing value proposi7on of OM5.

Table 1. Evolu[on of VCSEL-based links over MMF

Beyond400Gb/sEthernetOnce 100 Gb/s PAM VCSELs are available, when combined with two-wavelength BiDi transmission, 200G-SR1.2, 400G-SR2.2, 800G-SR4.2, and 1.6T-SR8.2 become feasible. While four-wavelength SWDM4 technology combined with the same VCSELs could support 400G-SR1.4, 800G-SR2.4, 1.6T-SR4.4. Wideband capability of OM5 makes it ideally suited for these mul7-wavelength future PMDs

While only some of the above PMDs will have adequate support/need to become either an IEEE 802.3 standard or MSA , 100 Gb/s PAM4 VCSELs lay the founda7on for VCSEL-MMF links suppor7ng 800G and 1.6T MAC rates as those rates become standardized in the future. And there is a CFI “Beyond 400 Gb/s Ethernet” that is scheduled for November 2020.

References:

1. Paul Kolesar, CommScope “400GBASE-SR8 MDI Choices” IEEE P802.3cm TF, May 2018 hHp://www.ieee802.org/3/cm/public/May18/kolesar_3cm_01_0518.pdf

2. “100 Gbps Port Shipments to Peak in 2020” Dell’Oro Group. August 2019 hHps://www.delloro.com/news/400-gbps-shipments-to-surpass-15-m-switch-ports-by-2023/

3. Robert Lingle, Jr, OFS, Mabud Choudhury, OFS “Lower cost, short reach, op7cal PHYs using 100 Gb/s wavelengths” IEEE 802.3 Call-for-Interest, November 2019. hHps://www.ieee802.org/3/cfi/1119_1/CFI_01_1119.pdf

4. Tom Issenhuth and Brad Booth, Microsoz “100G-DR Use Cases & End User Perspec7ve” IEEE P802.3cd TF, September 2016 hHp://www.ieee802.org/3/cd/public/Sept16/issenhuth_3cd_01a_0916.pdf

5. IEEE P802.3db PAR, CSD, Objec7ves, hHps://www.ieee802.org/3/db/index.html

6. Jonathan Ingham, Broadcom, Ramana Murty, Broadcom “Towards technical feasibility of 100 Gb/s per lane op7cal PMDs suppor7ng 100 m OM4 MMF” June 2020 hHps://www.ieee802.org/3/db/public/adhoc/presenta7ons/ingham_3db_adhoc_01a_062520.pdf