BNL PDN Enhancements

Perimeter Load Balancers

• Scaleable Performance• Fault Tolerance

• Server Maintainability• User Convenience

• Perimeter Security

Cisco Content Sensitive Switches

• Dual Cisco 11506 units for fault tolerance• Dual Cisco 4506 switches for proxies• Rated at 40GB/Sec. Maximum throughput• Virtualizes site perimeter services• Extreme scaleable and flexibility• High availability and redundancy

Content Switches cont.

• ACL based proxy service access (secure)• Provides expandable pools of servers and

services• Transparent to end users• A single IP address / DNS name for all

servers in the service pool (Virtual IP)• Load balanced user access to proxies based

on Least Number of Connections algorithm

Content Switches cont.

• Proxies assigned RFC 1918 (Private IP) space (additional isolation)

• Linear scalability• Individual servers can be added to or

removed from the service pool at will. This facilitates software upgrades, maintenance, and patch support for the actual servers.

CSS VIP Security

• Behavior similar to Pix Firewall

• Outbound traffic permitted by default• Inbound traffic subject to ACL optional

• Protects all pool services• Internet scans show no or minimal services

(Only the advertised services)

Performance Overview

• Services virtualized and “Pooled” together• Approximately Linear Scalability• / 28 for individual service pools 14 slaves max

• Separate management and load traffic paths

Proxy Services Virtual IP’s

• SMTP 1.1.1.1• HTTP 1.1.1.2• SSH 1.1.1.3• TELNET 1.1.1.4• HTTP/Reverse 1.1.1.5• FTP 1.1.1.6• Others as we grow

ESNET

NYSERnet

OC-12

GIG-ESD

C I S CO YS T EM SS

PIX Firewall SERIES

Pike PIX 535

BNL Perimeter Proxy - Upgrades

SD

Catalyst 6500CISCO YSTEMSS

SERIESÒ

Tefnutcat515-

13

CORE

BNL CAMPUS======>

BNL CAMPUS======>

Outside interfaceInside interface

SD


Ò

SER IES

C4506

SD


SERIESÒ

shucat515-

12

SD


SERIESÒ

Amoncat515-

9

Trunk

GIG-E

SD


Ò

SE RIES

CSS11503Load Balance

vl300

GIG-E

GIG-E

GIG-E

GIG-E

DS-3

Service Module

GIG-EGIG-ESD


Ò

S ERIE S

C4506

SD


Ò

SERI ES

CSS11503Load BalanceGIG-E

GIG-E

GIG-E

GIG-E

APP Trunk

<======Internet

Virtual ProxyFarms

ftp, telnet,ssh,smtp

Exampleeth0.310 Link encap:Ethernet HWaddr 00:03:47:DB:6D:6B inet addr:172.16.1.13 Bcast:172.16.1.15 Mask:255.255.255.240 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1945993 errors:0 dropped:0 overruns:0 frame:0 TX packets:214508 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:91180210 (86.9 MiB) TX bytes:14828768 (14.1 MiB)

Management Server Configuration

• IEEE 802.1q Trunk Format (LB Monitor Interface)

• Custom Linux Kernel Configuration Parameters

• Subset of NIC cards, Intel EEPro 100 with Intel Driver

• Vconfig utility to create vlan (IEEE 802.1q tag) interfaces

Performance Tests

single test

[SUM] 0.0-253.6 sec 15.2 GBytes 516 Mbits/sec

psudo double test

smtpvip2:~#iperf -c 198.124.238.14 -n 209715200 -t 300 -P5

------------------------------------------------------------Client connecting to 198.124.238.14, TCP port 5001TCP window size: 64.0 KByte (default)------------------------------------------------------------[ 5] local 172.16.129.66 port 32832 connected with 198.124.238.14 port 5001[ 6] local 172.16.129.66 port 32833 connected with 198.124.238.14 port 5001[ 7] local 172.16.129.66 port 32834 connected with 198.124.238.14 port 5001[ 8] local 172.16.129.66 port 32835 connected with 198.124.238.14 port 5001[ 9] local 172.16.129.66 port 32836 connected with 198.124.238.14 port 5001[ ID] Interval Transfer Bandwidth[ 8] 0.0-300.1 sec 1.89 GBytes 54.2 Mbits/sec[ 6] 0.0-300.1 sec 1.85 GBytes 53.0 Mbits/sec[ 5] 0.0-300.1 sec 1.87 GBytes 53.6 Mbits/sec[ 9] 0.0-300.2 sec 1.76 GBytes 50.3 Mbits/sec[ 7] 0.0-300.2 sec 1.84 GBytes 52.7 Mbits/sec[SUM] 0.0-300.2 sec 9.22 GBytes 264 Mbits/sec

[ ID] Interval Transfer Bandwidth[ 7] 0.0-300.1 sec 1.78 GBytes 51.0 Mbits/sec[ 9] 0.0-300.2 sec 1.86 GBytes 53.3 Mbits/sec[ 5] 0.0-300.7 sec 2.00 GBytes 57.0 Mbits/sec[ 8] 0.0-300.7 sec 1.68 GBytes 48.1 Mbits/sec[ 6] 0.0-301.0 sec 1.82 GBytes 52.0 Mbits/sec[SUM] 0.0-301.0 sec 9.14 GBytes 261 Mbits/sec

2 runs of a single machine in the VIP, 2 runs 2 machines in the VIP

Confirmation from different measuring tool

netmon:~# nmap -P0 1.1.1.1-5 Starting nmap 3.50 ( http://www.insecure.org/nmap/ ) at 2004-07-12 15:11 EDTAll 1659 scanned ports on csssm1 (1.1.1.1) are: filtered …...Interesting ports on smtpgateway (1.1.1.2):(The 1656 ports scanned but not shown below are in state: filtered)PORT STATE SERVICE25/tcp open smtp79/tcp open finger113/tcp open auth All 1659 scanned ports on httpgateway (1.1.1.3) are: filtered Interesting ports on cecache (1.1.1.4):(The 1655 ports scanned but not shown below are in state: filtered)PORT STATE SERVICE80/tcp open http443/tcp open https563/tcp open snews8080/tcp open http-proxy All 1659 scanned ports on 1.1.1.5 are: filtered

Summary• Cisco CSS provides a high throughput

scalable solution for most BNL perimeter services

• Security enhancements are additional features

IP v6

• Test Bed Deployment

• Campus Network and Host Security

•Low Cost

SD

Pike Cisco 535Firewall

NYSERNetESNet

Nephthys Cisco 6509Layer 3 Switch

Bldg. FeedsFE 100mb

Bldg. FeedsFE 100mb

BackboneGigabit

Backbone Gigabit

DS354MB

OC12622MB

SD

SCISCO YSTEMS

Amon Cisco 6509Layer 3 Switch

S D

SCISCO YSTEMS

SD

SCISCO YSTEMS

Anubis Cisco 6509Layer 3 Switch

SD

AC FAN OUTPUT

OK OK FAIL

I

AC FAN OUTPUT

OK OK FAIL

I

POWER

APOWER

B

0 1 2 3 4 5 RSP RSP 8 9 10 11 12

Pteh Cisco 7513Router

IPv6BackboneFE 100mb

BNL Campus Network

IPv6 TrunkFE 100mb

IPv6 WAN and CoreRouter

6to4 Link

IPv6 RedundantTrunk

FE 100mb

Figure 1 BNL IPv6 Core

• Built from “recycled” 7513 free

• Separate Infrastructure• IPv6 802.1q Trunk Encapsulation

• EUI-64 /64 subnets• HTTP and FTP servers

• Next Step: Fix DNS • NatPT or dual stack

BNL PDN Enhancements

Documents

Transcript of BNL PDN Enhancements