Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S....
-
Upload
shanon-crawford -
Category
Documents
-
view
217 -
download
0
Transcript of Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S....
![Page 1: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/1.jpg)
Network Architecture
Gary Buhrmaster
ST&E Readiness ReviewMay 14th, 2007
Work supported by U. S. Department of Energy contract DE-AC03-76SF00515
![Page 2: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/2.jpg)
Network Philosophy
Support getting the science done (safely) The science is the thing
Simplicity (where possible) Limit vendors, technologies used Leverage existing SCCS staff expertise
Redundancy (where appropriate) SCCS is not staffed for 24/7 coverage “Throwing smart (dedicated) people at issues”
works as long as you do not throw them too often
![Page 3: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/3.jpg)
Overview
SLAC administers globally routed network space of 134.79.0.0/16 “SLAC” address space Visitor and RAS subnets IPv6 (test) subnet
A number of internal private subnets for control systems, isolated systems, batch farms Accelerator, SSRL, IR2, SCCS
![Page 4: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/4.jpg)
Overview
Hardware Vendors: Cisco, Nokia ~300 Layer 2 (capable) devices ~50 Layer 3 (capable) devices ~20 Enforcement (firewall/filter) devices Many devices are categorized as more than one
swouters/frankenrouters (not all swouters are used as L2/3) what is an infiniband “switch” (it has routing in it…)
Misc. appliances (WLSE (HP), EndRun) ~15 support systems (logging, monitoring, etc.)
Sun/Dell – systems managed by the systems group
![Page 5: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/5.jpg)
Overview
Physical instantiation ~70 buildings
Some buildings have numerous switches (some none) klystron gallery, computer center, SSRL
~200 VLANS Switched network design Some buildings have multiple subnets/vlans Some vlans are in multiple buildings, some in only one
Some in only one switch router to router connections, span monitoring…
Some internally used by devices
![Page 6: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/6.jpg)
Staffing
Network Engineering Manage/Configure/Monitor network devices Five FTEs
Network Research Primarily research activities
But operationally focused (not just blue sky), which is leveraged to support SLAC and HEP/BES activities (especially WAN performance issues)
![Page 7: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/7.jpg)
Staffing (outside of Network group) Network Operations
Reports to SCCS Operations Physical installation/support Five FTEs
Netops also coordinate with CEF staff and contractors for some installations (cable pullers, bulk fiber installation and termination, etc.)
![Page 8: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/8.jpg)
Staffing (outside of Network group) Security group
Responsible for overall security policies and approvals
Apply approved policies to the Cisco enforcement devices
Windows group Apply approved policies to the Checkpoint
enforcement devices Systems group
Maintain the Unix network support systems
![Page 9: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/9.jpg)
SLAC Speak
IFZ – Internet Free Zone At least some part of every network is blocked
from offsite network access Printers, Batch nodes, Network devices, “problematic”
devices (i.e. SBCs/IOCs)
SFZ – SLAC Free Zone Some special networks (controls) are accessible
only from their local networks IR2, MCC
![Page 10: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/10.jpg)
SLAC Speak
RouterBlock Layer 3 forward and uRPF blocking (advertise
the /32 addresses into routing table to null route device at the router(s))
EPN – “Extremely Private Network” Elevated level protections (the “PII” place)
EPN(1) (original design), EPN2 (revised design)
CANDO – Computer And Network Database in Oracle (?)
Database of record for IP addresses/systems
![Page 11: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/11.jpg)
……………………………………………………
Big (dense) Picture
Border
MCCIR2 SSRL BSDEPN
VPN “Special”NetMgmt
Farm Netrsch
Infra Campus
Core
“Internet”IPv6
visitor
But still simplified
![Page 12: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/12.jpg)
Drill down (Layer 3 view)
Network segmentation Enclaves
SLAC, accelerator…. Functional/Physical
research yard, visitor network, decnet Performance/Availability
batch farm, network research
![Page 13: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/13.jpg)
IPv6 Network
Dipping a toe in the (IPv6) water its cold and lonely there
External to SLAC network One web server
was originally proposed to be named VVVVVV
ESnetBAMANrtr-ipv6
IPv6 Network
WWW
![Page 14: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/14.jpg)
Visitor (& RAS) Network
External to SLAC network (no trust) Wireless access is only on visitor network Client only support (block servers)
ESnet
BAMAN
Visitor Network
![Page 15: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/15.jpg)
Border Network
Border enforcement device is a filtering router ACLs block ports <1024 (except to allowed hosts), and
various special ports (X, netbus, backoriface, …)
ESnet
Stanford
CENICInternet2
BAMANBorderrouter
![Page 16: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/16.jpg)
Infrastructure Services
Centrally administered servers Windows/Unix infrastructure services
Unix & Windows infrastructure – DNS, Kerberos, AFS, AD, file servers, web services, email, ….
IFZ and where possible Most exceptions to port < 1024 filters are to these
servers (web, email, kerberos)
SLAC Network“Nethub/IFZ/IFZ-Lite”
B050(2nd floor)
![Page 17: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/17.jpg)
Campus
Most staff/engineers/scientists are connected to one of the “PUB” networks Legacy workgroup allocations (based on “yellow
cable”) have changed to physical location allocations (trying to avoid flat earth operations)
Campus
Campus Distribution
Access (many buildings)
![Page 18: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/18.jpg)
Farm
Batch resources for scientific discovery Most resources are IFZ
Exceptions for external data transfer systems, and scientific login systems
Many resources are (policy (i.e. netgroup)) limited to be used only from other batch systems
Different Availability/Performance needs
SLAC Network
“Farm” networks
batch systems
Campus
![Page 19: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/19.jpg)
BaBar / IR2
IR2 has four subnets one public general purpose subnet, one IFZ
subnet (local compute farm), one SFZ subnet (dedicated SBCs and detector subsystems) with EPICs gateway, and isolated device control
Intention is that these networks/systems can operate independently from SCCS
mcc
Farm
![Page 20: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/20.jpg)
Accelerator (MCC)
Accelerator network has four subnets One public general purpose subnet (slclavc), two “slac free”
subnets (leb, slcc) for control systems, and one isolated subnet (pep)
Use of multi-homed controls systems (VMS) for access to isolated networks devices
Intention is that these networks/systems can operate independently from SCCS
IR2
SLAC Network
![Page 21: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/21.jpg)
Network Management
Network monitoring and configuration management (BAM - Backup and Monitoring) SNMP (via acls on network devices) only respond
to requests from the management network hosts ACLs protect appliances/APs (bastion hosts) Systems are limited access
SLAC Network Network Management and monitoring networks
![Page 22: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/22.jpg)
Network Research
Network Research activities Isolated to allow local experimentation
ex: tsunami multicast
Systems are maintained the same as other systems on site
Systems are limited login, sponsored users
SLAC Network Research network
![Page 23: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/23.jpg)
SSRL
SSRL manages their own network equipment and configurations, including their own firewall implementations to protect their control and experimental systems A later presentation will discuss SSRL
![Page 24: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/24.jpg)
BSD (EPN(1))
EPN(1) Air Gap possibility Extensive filtering Users access PeopleSoft via Citrix More details in later presentation
rtr-bsdnet
bsd-epnbsd
SLAC net
bsd-dmz
![Page 25: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/25.jpg)
EPN2
Revised approach based on new PS arch Multiple DMZ nets (web servers), Backend nets
(app servers, DBs) In realty, collapsed firewalls
Details in later presentation
SLAC Network
DMZsBackend
![Page 26: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/26.jpg)
VPN
VPN (GRE/IPSEC) only to official servers Windows PPTP/L2TP VPN server Discouraged (use Citrix where possible) Firewall/filters
Block RPC, NFS, CIFS except to approved servers, & NetBus, BackOriface, etc.
SLAC Network VPN Servers
![Page 27: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/27.jpg)
“Special” subnet(letts)
A few networks specially protected due to inability to maintain the systems, or certified configurations Ex: GLAST Clean Room, PCD, HVAC Group responsible for equipment purchase, SCCS
maintains the devices/configurations
SLAC Network
SLAC Network
SLAC Network
![Page 28: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/28.jpg)
Procedures/Policies
Device connection policy Devices need to be in CANDO
Network equipment Users are not to install switches/routers/hubs
Wireless No wireless on the SLAC networks Devices installed/coordinated by SCCS
![Page 29: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/29.jpg)
Network protections
Dedicated subnet for network management Network devices are IFZ SNMP restricted to network management subnet
SSH on all but a few legacy devices Finally got funding to upgrade the last few
Disable ports not allocated on switches No devices on native .1q vlan WLSE used for rogue access point detection
![Page 30: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/30.jpg)
Network protections
Restricted physical access to “core” devices (Building 050 OmniLock door access)
Routing/switching best practices no ip unreachable, BGP passwords, schedule
allocate, no source route, …. Strong working relationship with upstreams
![Page 31: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/31.jpg)
Network Intrusion Detection
Primarily log and netflow based Central logging and analysis
“Significant” events cause paging Netflow detects many scanners (and P2P)
Collected for both internal and external traffic “scanning” detection catches (SMTP) bots in “real time”
And the occasional “special” user Extremely useful for incident analysis
![Page 32: Network Architecture Gary Buhrmaster ST&E Readiness Review May 14 th, 2007 Work supported by U. S. Department of Energy contract DE-AC03-76SF00515.](https://reader035.fdocuments.us/reader035/viewer/2022062517/56649f1c5503460f94c334dd/html5/thumbnails/32.jpg)
Discussion?
Obligatory final slide to avoid “End of slide show” artifact