Sumitomo Electric Lightwave_The 8 Fiber Solution for Today's Data Center_April 2015

©2015Sumitomo Electric Lightwave. All rights reserved.

The 8-Fiber Solution for Today’s Data Center

Bill Charuk, SR Product Manager, Data Center Solutions

Introduction

Data center infrastructure is constantly evolving. Fiber optic installations have used the

same basic design for decades. However, to fulfill today’s needs and be ready for

tomorrow’s higher speed standards, a new approach should be considered. The

innovative network architecture using 8 fiber ribbons as the basic infrastructure building

block instead of a 12 fiber subunit is examined to improve significantly data center

design, performance, futureproofing, and cost savings.

Background

Until now, data center infrastructures have been based on the tried and true designs

that have served the enterprise network so well for the last 30 years. This is not

surprising, since the equipment used and data rates involved in data center construction

were essentially extensions of existing enterprise network requirements. With the

advent of more and more internet connections and an explosion in social media and

mobile devices driving ever larger amounts of data around the internet, the speeds

necessary to move the data have progressed as well. When data speeds were increased

to 56Kbs for fax machines, most thought they would never need transmission beyond

this amazing speed. In 1985 IEEE 802.3 published the 10BASE2 standard for 10Mbs

over thin coax. In 1993 the 802.3j committee published 10BASE-F standards for 10Mbs

transmission over optical fiber. 1Gbs speeds were codified in 1998 with the adoption of

the 1000BASE-X Gbit/s standard by the IEEE802.3z committee. Link aggregation was

standardized in 2000 and 10Gbe in 2002.

Today data centers are firmly positioned in the 40Gbe/100Gbe world with 400Gbe on the

horizon. With this rapid increase in data rates, industry standards have adapted to

ensure not only that the network could function, but also maintain the ability to utilize

products and technologies that are cost effective and known to the end user market. In

http://en.wikipedia.org/wiki/1000BASE-X

http://en.wikipedia.org/wiki/Gigabit

©2015 Sumitomo Electric Lightwave. All rights reserved.

78 T.W. Alexander Drive, Research Triangle Park, NC 27709 • 800.358.7378 • www.sumitomoelectric.com 2

order to facilitate “graceful growth” in the data center, another infrastructure option

needs to be considered, one that can be retrofitted to existing networks, while

simultaneously supporting future growth needs and technology requirements.

What and Who is Driving Data Demand?

Connections to the internet are ubiquitous today, almost akin to carrier pigeons in New

York City at the turn of the century. In 2015 alone, 15 billion devices will be connected

to the internet and 3 billion users will access it. Faster broadband speeds, 28Mbps in

2015 versus just 7Mbps in 2010, will be considered standard. In addition, demands for

higher quality media and faster downloads, from a 1 minute video download in 2010 to a

2 hour HDTV movie in 2015, are adding to the need for higher velocity data downloads.

Cumulatively, each person that is or wants to be connected to the internet is driving

data demand, and consequently higher and higher data rates. Of course, a great deal of

data today is stored. If the data is stored, then it will need to be backed up. The

exponential growth of the amount of stored data will necessarily drive the need for

higher and higher speeds to perform these operations in a given amount of time.

1 nowell_01_0911.pdf citing Cisco Visual Networking Index (VNI) Global IP Traffic Forecast, 2010–2015,

High Speed Network Migration

As evidence that higher speeds are proliferating in the marketplace, Chart 1 shows the

recent trend in server shipments supporting 1/10/40Gbe over the last several years. By

the end of 2016, 1Gbe server port shipments will become insignificant in the

marketplace.

Chart 1

http://www.sumitomoelectric.com/



This migration will continue until the amount of data required to be transmitted over the

internet stops growing. As already shown, the use of the internet continues to grow for

a myriad of reasons and likely will not abate at any time in the near (and not so near)

future.

Chart 2 illustrates the number of port shipments for 40Gbe and 100Gbe ports over the

same time period.

2OFC-Panel-2-Need-For-Speed-Final Ethernet Alliance - http://www.ethernetalliance.org/event/ofc-2013/

As these charts clearly show, the number of high speed (>10Gbs) ports will grow

exponentially in the coming years. It should also be noted at this point that while

40GbE and 100GbE deployments (which will be viewed as equal technologies from an

infrastructure point of reference) are growing, the next step in data transmission

standards is underway; to 400Gbs.

Why is this important? To answer this question, we must first examine basic networking

requirements of 40Gbs networks and how, in some instances, they are fundamentally

different than those that operate at lower speeds.

High Speed Network Fundamentals

While this may not seem as a change any different than shifts to higher speeds that

have occurred previously, the change for 10Gbe network to a 40Gbe network is

Chart 22


http://www.ethernetalliance.org/event/ofc-2013/



markedly different, unlike any that has occurred in the past. Specifically, until the

advent of 40Gbs speeds, all optical transmission in premises/data center environments

used serial transmission schemes. That is, the data stream consisted of information

being sent one byte at a time, one after the other, at a given rate. Of course, the ability

to complete this transmission and have it be able to be discerned at the receiver is a

function of several system variables, most notably the speed at which the receiver is

able to transmit discreet light pulses that are readily detectable at the receiver.

As transmission speeds increase, serial schemes are problematic for Vertical Cavity

Surface Emitting Lasers (VCSELs). The ability of the VCSEL to transmit at higher

speeds, coupled with the distortion of signals by polarization mode dispersion and

chromatic dispersion in the optical pathway, negates use of current VCSEL’s at higher

speeds in serial transmissions. However, to use higher speed lasers (which are readily

available) would necessitate an unacceptable increase in the cost of transmission

equipment and essentially make obsolete all installed multimode optical fiber in data

center and enterprise networks.

The way to utilize inexpensive VCSEL technology and meet the speed requirement is to

use parallel optics. Or more simply put, aggregate several channels at lower speeds to

reach higher speeds required. This is precisely what the 40G/100G standards, as

adopted by IEEE 802.3ba-2010 committee, advises for networking technologies in the

transmission of Ethernet frames.

The standard, taken in its simplest form, identifies fiber optic link requirements for

transmission at 40/100Gbs over multimode fiber. See Table 1.

Table 1


http://en.wikipedia.org/wiki/Computer_network

http://en.wikipedia.org/wiki/Ethernet_frame



However, there are obvious differences in how these speeds are achieved versus

previous standards. For 10Gbs transmissions and below, serial transmission was used.

In figure 1, we see that the channel consists of 1 transmission link (TX) and one receiver

link (RX) both operating at 10Gbs. This is typically accomplished using a duplex LC

connector as shown.

Figure1-Serial Transmission

When using parallel path transmissions, the single lanes are multiplied to reach an

aggregate speed. As seen in Figure 2, TX and RX channels are now composed of 4 lanes

each. These lanes can be comprised of individual 10Gbs or 25Gbs lanes to reach 40Gbs

and 100Gbs respectively. Connectivity in this case is achieved by using MPO (Multiple-

Fiber Push-On/Pull-off) connectors. An MPO connector is pictured in Figure 3. These

connectors are designed to terminate and align 12 fibers simultaneously.

Figure 2-Parallel Lane Configuration

Figure3-MPO Connector




The 12 fiber MPO connectors have been in use in the network for some time and are well

known. However, there is an inherent problem with their use when installed in a parallel

path system. As noted from Figure 4, there are 4 TX, 4RX and 4 unused or “dark” fibers

per connection. While not a technical problem in and of itself, this scheme presents a

problem to the network operator.

Figure 4-MPO Channel Assignments for Parallel Transmission

The problem arises when a 40G/100G solution is required to be implemented using a 12

fiber infrastructure. As seen in Figure 4, there are 4 unused fibers in the center of the

connector related directly to every 12 fiber subunit in the cable/cables being used. This

equates to a 33% loss in the fiber network utilization. The loss of 33% of the physical

network can be a costly concern. While there are ways to overcome this issue, which

will be explored later, they do not always result in the most cost effective or best optical

pathway for the data center network.

Essentially, with the new standards, parallel transmission schemes provided for by the

IEEE standards clearly identify the need for an eight fiber based solution. This is a

paradigm shift from the way networks have been designed and installed for the past 30

years in enterprise networks and in most data centers.

Physical Layer Solutions…Getting to 8 Fibers

The first eight fiber solution must encompass existing networks. The question at hand is

how we achieve use of all the fibers in an existing 12 fiber based network. This

methodology is well known and readily available. The basic premise of this “work

around” is that the existing 12 fiber groups, usually with a minimum fiber count of 24,

are rearranged into a subset of 8 fiber groupings. Therefore 24 fibers in two 12 fiber




groupings are rearranged into three 8 fiber groups, thereby using all the fibers in the

network.

This can be accomplished practically in one of two ways. The first is to use a specifically

designed cable harness that has two 12 fiber connectors on one end and three 12 fiber

MPO connectors on the other end with only 8 fibers populated in the necessary locations

to connect the parallel channels. This method has a disadvantage in that the harnesses

can be unwieldy and difficult to dress into the fiber termination hardware due to the

transition from two subunits to three.

A more popular method is shown in Figure 5. In this scenario, the conversion occurs in

a cassette/conversion module, which is mounted in a standard fiber optic patch panel.

Note the two 12 fiber MPO connections at the rear and three 8 fiber MPO connections at

the front of the module. This can also be accomplished with a single 24 fiber MPO at the

rear of the module.

Figure 5-8 Fiber Conversion Cassette

Figure 6 below (next page) shows how this is accomplished internal to the

cassette/conversion module.




Figure 6

It must also be remembered that when installing this conversion on a 12 fiber

infrastructure, this conversion must occur on both the TX and RX sides of the channel,

so the additional costs for the two conversions must be added to the planning of a

40/100G network for every link.

The question may arise, “if technology exists that can convert the ubiquitous 12 fiber

infrastructure to parallel transmission standards, why do I need to consider an 8 fiber

infrastructure?” There are two areas that need to be examined further that will explain

why the 8-Fiber Solution makes more sense. These will be highlighted as we discuss the

8- Fiber Solution further.

8 Fiber Infrastructures for Data Center Applications

The eight fiber solution for data centers is essentially a tailored network infrastructure

designed to support 40/100G (and 400G) networks in the most efficient and cost

effective manner. The major difference is obviously that the subunits of the cable

infrastructure are based on 8 fiber count instead of 12.




By designing a network using the 8 fiber subunits as the building blocks, direct

connections without conversions are now achieved. The only item in which care must be

taken is the connection at the end equipment — the 8 fibers terminated in the

connecting MPO connector must be aligned as shown in Figure 4, with the operating

fiber separated by four “blank” fiber spaces (Note: this space is necessary so the fibers

can attach to the TOSA and ROSA in the end equipment transceiver). This segregation

can be easily made using a preterminated patch cord or a field installable Splice-On

MPO connector of the correct type (i.e. one specifically designed for this application).

There are several advantages to this type of installation. The first is cost. By installing

a 12 fiber system for 40/100G systems, the operator will incur costs for the conversion

at either end of the system. Depending on the method used, these costs can be over

$100 per port. In a large installation, these costs can be a significant part of the overall

fiber build.

The main advantage from a technical standpoint requires review of the standards for

attenuation as shown in Table 1. This time, focus needs to be not on the distances

allowed, but the Maximum Channel Connector Loss.

Reverting to the discussion surrounding using 12 fiber subunits and the conversion

required, in either method of conversion, an extra connection is introduced on both the

TX and RX ends. The conversion method adds TWO extra connector loss points to the

optical pathway. Link loss savings can be 0.2 to 0.5dB by removing these two

connections! As we can see above, the longer the distance, the less connector loss is

allowed in the link. In order to achieve the lowest optical loss in the system, the 8-

Fiber Solution should be employed.

How is the 8 Fiber Solution Implemented?

We have already shown that the connectivity portion of this solution is accomplished by

use of 8 fiber MPO connectors designed for use in parallel path systems; however the

cable subunit construction is the major consideration and the largest departure from

typical enterprise structures. Cables with subunits built on 8 fiber multiples need to be




employed. For smaller fiber count cables, this is easily achieved by substituting 8 fibers

in a buffer tube or under a single jacket to achieve this goal. As fiber counts climb,

however, this can become problematic from an installation standpoint. To achieve

higher fiber counts in loose tube cables, more buffer tubes must be employed to hold

the requisite number of fibers, resulting in larger and larger cables. Additionally, these

cables typically have a larger diameter than corresponding high fiber count ribbon

cables.

It is difficult to put multiples of 8 fibers in a loose tube cable for a number of reasons.

First, most cables have been qualified both mechanically and in accordance with UL

testing with up to, but not exceeding, 12 fibers per tube. Additionally, industry color

code standards would necessitate up to half of the fibers to be striped. This would make

it difficult and time consuming to terminate the fibers correctly in the field.

What is the answer then? The best cable solution for 8 fiber systems is ribbon cables.

Optical fiber ribbon cables have been in use for many years and are available in the

same NEC categories as loose tube cables. Due to the unique construction of ribbon

cables, most have been qualified with ribbons based on 8 fiber counts, as well as the

more well-known 12 fiber counts. For higher fiber count cables especially (usually >96

fiber) 16 fiber, 24 fiber, and 32 fiber ribbons are available. In some cases, the ribbons

can be separated in the field into 8 fiber component ribbons. This feature markedly

improves productivity while doing terminations in the field since ribbonizing, as would be

required for loose tube termination, is not necessary. Figure 7 shows the cross section

of a 96 fiber ribbon cable with six, 16 fiber ribbons. Notice that in this arrangement, the

manufacturer has encapsulated two 8 fiber ribbons into sixteen, making them easy to

separate. Also of note is that the color code of the fibers is consistent once the 8 fiber

subunits are isolated, making it fast and easy to terminate.

Figure 7




Cables made up of 8 fiber ribbons are available in outside plant, plenum, riser and

indoor/outdoor versions with every fiber type included. Figure 8 shows the various

layers of encapsulation that make up a 16 fiber ribbon.

Figure 8

So what? I am only running 1Gbe today

Granted, some data center administrators and operators are planning on running only

1Gbs or perhaps 10 Gbs systems in their new data centers. However, 8 fiber

infrastructures will support these networks completely. Additionally, if at any time in the

future of the data center, should speeds need to be increased, the passive subsystem

already in place need not be altered at all. New end equipment can merely be attached

and the system is ready to operate at 40Gbs, 100Gbs, and beyond.

What about the future?

While there are no guarantees that the network infrastructure will evolve into something

completely different than that of today, it is probably safe to assume that future

technologies will not be of the disruptive variety in order to allow for graceful growth of

the network and to take advantage as much as possible of the installed base. In fact,

400G networks, the next probable step in higher speeds in the network, will easily be

supported by the same 8 fiber based solution discussed by using 8 X 25Gbs lanes or 4 X

100Gbs lanes. In many ways, the 8-Fiber Solution is the best technology option for any

data center for today and tomorrow.

Conclusion

While removal of existing 12 fiber subunit infrastructure networks solely for the purpose

of installing an 8 fiber 40G/100G solution is an alternative to complete renovation of the

data center infrastructure, it is clear that for new installations, which will now or can in

the future be called upon to support parallel path transmissions, it is the solution of

choice. Not only is the 8-Fiber Solution more cost effective by removing the need for

conversion products at either end of the optical link, but it also provides more optical

head room by removing two unneeded connections from the system.




By eliminating unused fibers, the unnecessary connectivity points, and conversion

module hardware, the 8-Fiber Solution ultimately yields both the best possible optical

performance and cost savings for your data center.

Moreover, the 8-Fiber Solution optimizes 40/100G and beyond, while backward

compatible with legacy networks (1G & 10G), creating a real-time, scalable, and already

future-proofed network.




78 T.W. Alexander Drive, Research Triangle Park, NC 27709

www.sumitomoelectric.com

800.358.7378



Sumitomo Electric Lightwave_The 8 Fiber Solution for Today's Data Center_April 2015

Documents

Transcript of Sumitomo Electric Lightwave_The 8 Fiber Solution for Today's Data Center_April 2015