CFP380 FCIP Theory - Brocade Community...
Transcript of CFP380 FCIP Theory - Brocade Community...
![Page 1: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/1.jpg)
FCIP Theory
Revision 0210
CFP380
1
![Page 2: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/2.jpg)
FCIP Theory
Revision 0210
CFP380
2
![Page 3: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/3.jpg)
FCIP Theory
Revision 0210
CFP380
3
![Page 4: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/4.jpg)
Footnote 1: Distances that utilize native FC can span 500km; these solutions incorporate dark fiber, C/DWDM, and form a single fabric.
For additional FCIP details, reference RFC 3821 – Fibre Channel Over TCP/IP (FCIP).
Brocade does not recommend FCIP for use in every distance extension scenario: no technical solution can be all things to all people. FCIP has inherent performance, reliability, data integrity, and manageability limitations when compared to native FC solutions. Delay and packet loss may create bottlenecks in IP networks. FCIP can support very long distances, as long as the carrier network is extremely high performance and reliable. FCIP is typically deployed when long-haul applications are not business critical, and do not need especially high performance. FCIP may not be suitable for tape, since tape usage will often fail if packets are dropped. In addition to its performance limitations, FCIP troubleshooting and performance analysis requires evaluating all aspects of the IP LAN and WAN networks in addition to all FC nodes, switches, and routers, which can make it more complex to manage than other extension options.
FCIP Theory
Revision 0210
CFP380
4
![Page 5: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/5.jpg)
FC traffic over an IP network
• Interconnection of islands of Fibre Channel storage area networks over IP-based networks
• Distance Extension over IP LAN/WAN/MAN
FCIP Theory
Revision 0210
CFP380
5
![Page 6: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/6.jpg)
After FC frames destined for devices at the remote side are encapsulated into TCP packets, a standard IP header is added to each packet. The packet is then sent to the next hop (usually an Ethernet router).
FCIP Theory
Revision 0210
CFP380
6
![Page 7: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/7.jpg)
FCIP Theory
Revision 0210
CFP380
7
![Page 8: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/8.jpg)
FCIP Theory
Revision 0210
CFP380
8
![Page 9: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/9.jpg)
FCIP Theory
Revision 0210
CFP380
9
![Page 10: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/10.jpg)
Footnote 1: An Upgrade License is required to enable all 16 FC ports and 6 GbE ports.
Footnote 2: Two GbE ports can be configured for copper (via built-in RJ45) or optical (via SFP module) cable connectivity. The other four GbE ports offer optical SFP module only. SFP ports will also accept 1 Gbps copper SFPs, Avago part number ABCU-5700RZ-BR1.Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP.
FCIP Theory
Revision 0210
CFP380
10
![Page 11: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/11.jpg)
Footnote 1: The top two GbE ports (GE0 and GE1) are configured for copper via built-in RJ45. In standard configuration, users have the option of using either the top two GbE ports, which are configured for copper SFPs, or the bottom two, left-most, GbE ports (GE0 and GE1) which are configured for optical SFPs. The remaining four GbE ports can use either optical or copper via SFP module up to 1 Gbps. It is possible to configure GE0 as copper and GE1 as optical and vice versa.
FCIP Theory
Revision 0210
CFP380
11
![Page 12: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/12.jpg)
Footnote 1: Fabric OS identifies power supplies left to right as Power Supply #1 and Power Supply #2. The Brocade 7800 requires a minimum of one power supply (connected and powered on) for switch operation. Footnote 2: Fabric OS identifies the fan assemblies left to right as Fan #1 and Fan #2.
Weight: 9.3 kg (20.6 lb)1Dimensions:Width: 42.87 cm (16.88 in)
Height: 4.30 cm (1.70 in) or 1UDepth: 61.0 cm (24.02 in)
Power Consumption: 84 Watts nominal, 91 Watts maximum
Both weight and power consumption numbers assume Brocade 7800 with two power supply/fan FRUs and zero SFPs installed.
FCIP Theory
Revision 0210
CFP380
12
![Page 13: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/13.jpg)
The base unit has 2 GbE ports available for use. They can be the two copper ports, or the two fibre ports. The default is copper.
The upgrade license is named the 7800 Upgrade license.
FCIP Theory
Revision 0210
CFP380
13
![Page 14: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/14.jpg)
Please refer to the Release Notes for the most up-to-date information.
Enabling either of the 2 x10 GbE ports requires a 10 GbE license, which is a slot-based license. Footnote 1: The supported operational modes are:
• 10 x 1 GbE port
• 10 x 1 GbE ports and 1 x 10 GbE port• 2 x 10 GbE ports
FCIP Theory
Revision 0210
CFP380
14
![Page 15: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/15.jpg)
Each Cavium processor has 4G of DDR memory.
FCIP Theory
Revision 0210
CFP380
15
![Page 16: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/16.jpg)
Footnote 1: Typically, port groups for Condor2 ASICs are 8-port trunk groups. There are 40 ports on a Condor2 ASIC. As seen in the notes section of the previous slide, there is a 5-port trunk to each of the four Blaster FPGAs. By definition, a trunk must be created from the same octet. This means that each octet from a Blaster trunk octet has 3 ports remaining in the octet. Thus, Brocade engineers used the available ports to create 2 other 2-port trunks to use some of the remaining ports.
Weight: 3.2 kg (7.0 lb)1Dimensions:
Width: 3.60 cm (1.41 in)Height: 42.06 cm (16.56 in)
Depth: 29.89 cm (11.77 in)Power Consumption: 235 Watts nominal Approximate weight with 0 SFPs installed. The minimum power consumption of the Brocade FX8-24 Extension Blade is 235 watts with 0 optical SFPs installed running at 8 Gbps.
FCIP Theory
Revision 0210
CFP380
16
![Page 17: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/17.jpg)
Footnote 1: Requires the 10Gbps license.
FCIP Theory
Revision 0210
CFP380
17
![Page 18: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/18.jpg)
FCIP Theory
Revision 0210
CFP380
18
![Page 19: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/19.jpg)
Footnote 1:
FCIP Tunnels: •B7800 - up to 8 VE_Ports
•FX8-24 - up to 20 VE_PortsFCIP Trunking:
•B7800 - up to 4 circuits per trunk•FX8-24 - up to 4 circuits per trunk for 1 GbE; up to 10 circuits per trunk for 10 GbE
Footnote 2: 10 GbE, Advanced Extension and FICON Accelerator licenses for FX8-24 supported in DCX and DCX-4S are slot based licenses
Footnote 3: Not supported in initial Fabric OS v6.3.0 release.
FCIP Theory
Revision 0210
CFP380
19
![Page 20: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/20.jpg)
Certified (Support is through the SFP vendor)
1/2/4 Gbps ELWL - 50km
1/2/4 Gbps ELWL - 100km
1/2/4 Gbps CWDM - 1470nm - 80km
1/2/4 Gbps CWDM - 1490nm - 80km
1/2/4 Gbps CWDM - 1510nm - 80km
1/2/4 Gbps CWDM - 1530nm - 80km
1/2/4 Gbps CWDM - 1550nm - 80km
1/2/4 Gbps CWDM - 1570nm - 80km
1/2/4 Gbps CWDM - 1590nm - 80km
1/2/4 Gbps CWDM - 1610nm - 80km
1/2/4 Gbps DWDM 80km (25 wavelengths from 1530nm-1560nm)
2/4/8 Gbps CWDM 40km(8 wavelengths from 1290nm-1430nm)
For the most up-to-date support, refer to the following site:
http://www.brocade.com/products-solutions/products/tranceivers/index.page
FCIP Theory
Revision 0210
CFP380
20
![Page 21: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/21.jpg)
FCIP Theory
Revision 0210
CFP380
21
![Page 22: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/22.jpg)
FCIP Theory
Revision 0210
CFP380
22
![Page 23: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/23.jpg)
C2 and GE2 = Condor2 and GoldenEye2 ASICs
FCIP Theory
Revision 0210
CFP380
23
![Page 24: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/24.jpg)
FCIP Theory
Revision 0210
CFP380
24
![Page 25: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/25.jpg)
Footnote 1: FC flow control mechanisms include R_RDYs and ACKs. FC communications also utilize long distance modes, BB credits, and VC channels.
FCIP Theory
Revision 0210
CFP380
25
![Page 26: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/26.jpg)
Footnote 1: There are 8 VE_Ports for 6 physical ge ports on the 7800 16/6. An FX8-24 blade can support 20 VE_Ports, and therefore 20 FCIP tunnels. Each FCIP tunnel is associated with a specific VE_Port. On FX8-24 blades, and on the 7800 switch, VE_Ports do not have to be associated with a particular GbE port.VE_Ports 12 through 21 may use GbE ports ge0 through ge9, or they may use XGE port 1. VE_Ports 22 through 31 can only be used by XGE port 0. The total bandwidth cannot exceed 20 Gbps.
FCIP Theory
Revision 0210
CFP380
26
![Page 27: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/27.jpg)
Footnote 1: As of Fabric OS v6.3.0, VEX_Ports are not supported on the FX8-24.
FCIP Theory
Revision 0210
CFP380
27
![Page 28: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/28.jpg)
The 7500E and 7500 switches and the FR4-18i blade support only one connection per GbE port, so strictly speaking, the FCIP circuit concept does not apply
FCIP Theory
Revision 0210
CFP380
28
![Page 29: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/29.jpg)
VE ports and GbE ports are no longer 1:1 associated as they were on the 7500 and FR4-18i.
VE and GbE ports can have a 1:1 association, but they are not limited by the design.
Footnote 1: Configuring a tunnel with more than one circuit requires an Advanced Extension license. Without a license present, a second circuit will not be allowed to be configured. The administrator will receive a message stating as such.
FCIP Theory
Revision 0210
CFP380
29
![Page 30: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/30.jpg)
FCIP trunking requires multiple FCIP circuits, therefore FCIP trunking cannot be implemented on the 7500E, 7500 switch, and the FR4-18i blade.
Footnote 1: Configuring a tunnel with more than one circuit requires an Advanced Extension license. Without a license present, a second circuit will not be allowed to be configured. The administrator will receive a message stating as such.
FCIP Theory
Revision 0210
CFP380
30
![Page 31: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/31.jpg)
Time out values:
• Default FICON is 1 sec• Default FC is 4 sec
• Depending on the solution being extended, the time out values may need to be changed from default.
FCIP Theory
Revision 0210
CFP380
31
![Page 32: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/32.jpg)
Example tunnel/circuit creation:
A tunnel using VE_Port 16 is created with an initial circuit 0 with a maximum rate of 1Gbps, metric of 0.
• IPPM_LINK_UP.bandwidth = 1GbpsCircuit 1 is added to Tunnel 16 with a max rate of 500Mbs, metric 0.
• TUNNEL_UPDATE.bandwidth = 1.5 GbpsCircuit 2 is added to Tunnel 16 with a max rate of 1Gps, metric 1.
• Since this is a higher metric, it is considered a standby and no TUNNEL_UPDATE is generated.
Circuit 3 is added to Tunnel 16 with a max rate of 1Gps, metric 1.• Since this is a higher metric, it is considered a standby and no
TUNNEL_UPDATE is generated.
FCIP Theory
Revision 0210
CFP380
32
![Page 33: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/33.jpg)
Example circuit failures/recovery (after initial creation state)
Circuit 1 fails.• Since this was a low metric circuit (active) we will try to re-establish, no
TUNNEL_UPDATE is sent to CP (and no updated will be sent once it re-establishes).
Circuit 0 fails, 1 is still failed.
• Since this is the last of the low metric circuits, the tunnel will failover to the next lowest circuits, which are circuits 2 and 3. Since the combined bandwidth of circuits 2 and 3 is different than circuits 0 and 1, a TUNNEL_UPDATE.bandwidth = 2.0 Gbps is generated to the CP.
Circuit 0 and 1 come back on line.• Since these are again the low metric circuits, we will resume traffic on these
circuits and 2 and 3 become the hot standbys. Since the bandwidth has changed again, a TUNNEL_UPDATE.bandwidth = 1.5 Gbps is generated to the CP. Note that the bandwidth was reduced.
Example circuit deletions (after initial creation state);
Circuit 1 is deleted.• Since this was a low metric circuit, all traffic will be directed to circuit 0. Again,
the bandwidth has changed so the tunnel will generate a TUNNEL_UPDATE.bandwidth = 1.0 Gbps to the CP.
Deleting circuits 2 or 3 at this point would have no affect since they are standby and the bandwidth is not included in the calculations.
Example circuit metric changes (after initial creation state);
Circuit 1’s metric is changed to 1.• Since this was a low metric circuit, all traffic will be directed to circuit 0. The
bandwidth has changed, so the tunnel will generate a TUNNEL_UPDATE.bandwidth = 1.0 Gbps to the CP. This would basically have the same affect on bandwidth as if deleting the circuit instead.
Circuit 3’s is changed to metric 0 (circuit 1 still has a metric of 1)• Since this was a high metric circuit changed to a low. The bandwidth has
changed, so FTNL will generate a TUNNEL_UPDATE.bandwidth 2.0 Gbps to the CP (circuit 0 and circuit 3 are now “active”, circuits 1 and 2 are now in “standby” mode).
FCIP Theory
Revision 0210
CFP380
33
![Page 34: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/34.jpg)
Footnote 1: Each 10 GbE port is assigned to a different FCIP subsystem. Because of this, tunnels cannot be created traversing the two ports. All circuits that reside in a tunnel must belong to the same FCIP subsystem.
FCIP Theory
Revision 0210
CFP380
34
![Page 35: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/35.jpg)
FCIP Theory
Revision 0210
CFP380
35
![Page 36: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/36.jpg)
3 Modes of operation for the Ethernet ports on FX8-24:
1G Only Mode: 10 x 1 Gbps ports are available for use (GE0 through GE9)10G Only: 2 x 10 Gbps ports are available for use (XGE0 and XGE1)1
Dual Mode: 10 x 1 Gbps ports and 1 x 10 Gbps port available for use (GE0 through GE9, XGE0)
FCIP Theory
Revision 0210
CFP380
36
![Page 37: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/37.jpg)
Footnote 1: IP address subnets are used so that the TCP supervisor can route packets to the destination subnets. The FCIP subsystem contains an IP routing table that directs packets with a tunnel destination to the correct circuits and tunnels.Footnote 2: The B7800 contains 6 x 1 Gbps connections between field-programmable gate arrays (FPGAs) and the Ethernet ports.Footnote 3: If multiple circuits are configured on the same physical port, they can contain addresses from the same subnet or different subnets.
Footnote 4: For the FX8-24, each FCIP complex has ten 1 Gbps connections between FPGAs and the Ethernet ports. This is the reason that the XGE ports require 10 circuits to achieve 10 Gbps bandwidth.
FCIP Theory
Revision 0210
CFP380
37
![Page 38: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/38.jpg)
• Packet loss re-transmissions are compounded when errors are bursty. Selective Acknowledgement (SACK) is an extension to a protocol which allows the acknowledge reception of specific packets or messages.The SACK option RFC 2883 [18] allows the receiver to acknowledge multiple lost packets in a single ACK, enabling faster recovery. An FCIP Entity may negotiate use of TCP SACK and use it for faster recovery from lost packets and holes in TCP sequence number space.Footnote 1: SACK improves loss detection, retransmission techniques, and enables faster recovery.
FCIP Theory
Revision 0210
CFP380
38
![Page 39: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/39.jpg)
Hardware compression is performed at FC ingress and de-compression is performed at FC egress.
Compression before encapsulation allows the TCP and FCIP headers to be visible on the network.
FCIP Theory
Revision 0210
CFP380
39
![Page 40: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/40.jpg)
Footnote 1: A value of 1 enables hardware compression. The 7800 switch provides two additional levels of software compression. Settings 2 and 3 provide incrementally higher compression ratios that can be used to improve performance on slower links. A value of 0 disables compression.
FCIP Theory
Revision 0210
CFP380
40
![Page 41: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/41.jpg)
FCIP Theory
Revision 0210
CFP380
41
![Page 42: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/42.jpg)
FCIP Theory
Revision 0210
CFP380
42
![Page 43: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/43.jpg)
FCIP Theory
Revision 0210
CFP380
43
![Page 44: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/44.jpg)
FCIP Theory
Revision 0210
CFP380
44
![Page 45: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/45.jpg)
FCIP Theory
Revision 0210
CFP380
45
![Page 46: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/46.jpg)
Ports ge4 and ge9 are available to be used, but are not in the example above.
FCIP Theory
Revision 0210
CFP380
46
![Page 47: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/47.jpg)
FCIP Theory
Revision 0210
CFP380
47
![Page 48: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/48.jpg)
FCIP Theory
Revision 0210
CFP380
48
![Page 49: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/49.jpg)
Best practice is to use a commit rate that uses 90% of the bandwidth allocated to the FCIP traffic. In this example, both devices are configured for 100% of the total bandwidth of the WAN gateway, which means that the router is oversubscribed 2:1. This configuration can lead to many dropped frames and errors.
FCIP Theory
Revision 0210
CFP380
49
![Page 50: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/50.jpg)
The configuration above is a better solution than the previous slide, but still has shortcomings. While the router is not oversubscribed, the bandwidth available for the WAN is not fully utilized when there is either a failure of one of the devices, or a simple case of a device needing less bandwidth than what is configured. This can leave the link underutilized during certain times.
FCIP Theory
Revision 0210
CFP380
50
![Page 51: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/51.jpg)
Footnote 1: Best practice dictates setting maximum commit rates to 90% of physical connection.
The best solution would to use Adaptive Rate Limiting (ARL) to better utilize the available WAN bandwidth. By configuring each device with a minimum commit rate of 50% of the allocated bandwidth, each device is then guaranteed a minimum amount of bandwidth at all times. If an outage occurs in one of the devices, or a device is not using all of its minimum committed rate bandwidth, the other active device can utilize the extra bandwidth up to its configured maximum commit rate.
FCIP Theory
Revision 0210
CFP380
51
![Page 52: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/52.jpg)
FCIP Theory
Revision 0210
CFP380
52
![Page 53: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/53.jpg)
FCIP Theory
Revision 0210
CFP380
53
![Page 54: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/54.jpg)
FCIP Theory
Revision 0210
CFP380
54
![Page 55: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/55.jpg)
Brocade’s original virtual channel model (pre-Condor2/GoldenEye2) divides an ISL into 8 virtual channels to insure that traffic of multiple priorities can travel across the link at the same time, without being disrupted, or disrupting other traffic.
���������������� �����������
VC # Assigned to
0 Class F
1 Class 2 Ack/Link Control
2 Data
3 Data
4 Data
5 Data
6 Class 3 Multicast
7 Broadcast/Multicast
All Data Traffic
FCIP Theory
Revision 0210
CFP380
55
![Page 56: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/56.jpg)
The QoS feature only comes into play if there is contention on the link. If there is no congestion on the link QoS will not engage.
The order of operations during congestion is as follows and repeats as necessary:1. VC0 then, 2. 6 frames of High priority traffic then,
3. 3 frames of Medium priority traffic then, 4. 1 frame of Low priority traffic
Results of QoS on a congested link:
FCIP Theory
Revision 0210
CFP380
56
![Page 57: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/57.jpg)
FCIP Theory
Revision 0210
CFP380
57
![Page 58: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/58.jpg)
Footnote 1: TCP port used is 3225.
FCIP Theory
Revision 0210
CFP380
58
![Page 59: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/59.jpg)
FCIP Theory
Revision 0210
CFP380
59
![Page 60: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/60.jpg)
FCIP Theory
Revision 0210
CFP380
60
![Page 61: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/61.jpg)
FCIP Theory
Revision 0210
CFP380
61
![Page 62: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/62.jpg)
FCIP Theory
Revision 0210
CFP380
62
![Page 63: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/63.jpg)
FCIP Theory
Revision 0210
CFP380
63
![Page 64: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/64.jpg)
FCIP Theory
Revision 0210
CFP380
64
![Page 65: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/65.jpg)
FCIP Theory
Revision 0210
CFP380
65
![Page 66: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/66.jpg)
FCIP Theory
Revision 0210
CFP380
66
![Page 67: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/67.jpg)
FCIP Theory
Revision 0210
CFP380
67
![Page 68: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/68.jpg)
FCIP Theory
Revision 0210
CFP380
68
![Page 69: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/69.jpg)
FCIP Theory
Revision 0210
CFP380
69
![Page 70: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/70.jpg)
FCIP Theory
Revision 0210
CFP380
70
![Page 71: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/71.jpg)
FCIP Theory
Revision 0210
CFP380
71
![Page 72: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/72.jpg)
FCIP Theory
Revision 0210
CFP380
72
![Page 73: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/73.jpg)
FCIP Theory
Revision 0210
CFP380
73
![Page 74: CFP380 FCIP Theory - Brocade Community Forumscommunity.brocade.com/.../CFP380-WBT_4_FCIP-Theory.pdf · Footnote 3: Link speeds of 1 Gbps require a 4 Gbps Brocade-branded SFP. FCIP](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a9f6fdd7f8b9a67178cd915/html5/thumbnails/74.jpg)
FCIP Theory
Revision 0210
CFP380
74