Network Planning Task Force - University of Pennsylvania · NPTF 2014 Schedule University of...
Transcript of Network Planning Task Force - University of Pennsylvania · NPTF 2014 Schedule University of...
University of Pennsylvania
Network Planning Task Force
September 8, 2014
Deke Kassabian [email protected]
NPTF Membership
Daniel Alig / Joe Cruz, Wharton
Charles Buchholtz, SEAS
Lena Buford / Tejash Patel, Annenberg
Cathy DiBonaventura, School of Design
Brian Doherty, SAS
David Domico, SRFS
John Eckman, Housing Services
Jeff Fahnoe, Dental
Mike Herzog / Didi Sariyska, GSE
Andre Jenkins / Michael Morris / Robert Colligon, PSOM
Marilyn Jost / Victoria Iannotta, FRES
Sue Kennedy, Business Services
David Kern, Public Safety
Kay McDonnell / Christine Droesser, Law
Grover McKenzie, Library
Donna Milici / Sam Smith, Nursing
Dominic Pasqualino, OACP
Smith Ragsdale / Brian Sherman, VET
Joseph Shannon, Finance
Mary Spada, VPUL
Marilyn Spicer, College Houses
Michael Weaver, Budget Mgmt. Analysis
Ira Winston, SEAS, SAS, Design
ISC Representation
Deke Kassabian, ISC (Chair)
Mark Wehrle, ISC
Jim Choate, ISC
Josh Beeman, ISC
Michel van der List, ISC
Mark Aseltine / Amy Phillips, ISC
Gary Delson / Geoff Filinuk, ISC
2 University of Pennsylvania - Network Planning Task Force
NPTF 2014 Schedule
3 University of Pennsylvania - Network Planning Task Force
July 21st –
• Information Security Update
September 8th –
• Network and Server Infrastructure (other than wireless)
October 13th –
• Wireless, Identity and Access Management, Penn+Box, Next Gen WWW
November 10th –
• Any remaining content
• Working through FY16 Rates
CSF and Port Rental / Maintenance
IP charges are 20% of the CSF
Headcount charges are 80% of the CSF
FY'14 FY'15
IP Charges: $1.61 / month $1.57 / month
FY'14 FY'15 10Base-T $4.75/month $4.75/month
100Base-T $4.75/month $4.75/month
1000Base-T $7.00/month $4.75/month
10 GbE $80.00/month $80.00/month
PoE ports $2.00/month $0.00/month
ISC Branding, Marketing & Website Project 4
Initial CSF Rate Change Information
Assuming an overall 2.75% staff salary
increase, and holding all else constant, costs
funded through CSF would increase by
roughly 1.45% for FY16.
This is NOT a rate announcement, just initial
information. Other factors will affect the
eventual rate.
Rate discussion will continue throughout the
fall meetings, and we will revisit the
recommended rate at the November 10th
meeting.
5 University of Pennsylvania - Network Planning Task Force
FY15
CSF Rate
$1.57/IP/Mnth
FY16
CSF Rate
Not yet set
FY’15 CSF Bundle of Services
Campus Backbone Infrastructure
Internet and Internet2 access
Rate limits on ResNet
IPv6, Multicast, and Advanced Networking
Public Wireless Subsidy
Cap on billable wireless IPs
NAP Operations, Fiber &Cable Mgmt
NOC Services and Network Management
Penn's Main Web and Central Pages
Online Directory and LDAP access
Classlists and SMTP Mail Relay
University Calendar Service
Infrastructure Services (DNS, DHCP, NTP
Penn+Box Storage & Collaboration
Network Address Translation (FY15)
Eduroam and IoT Support (FY15)
Enterprise Social Networking (FY15)
Security/ID Management
Kerberos, KITE, RADIUS
Penn WebLogin (CoSign and Shibboleth)
The InCommon Federation
Enterprise InCommon Cert Service
Authorization (Penn Groups)
PennNames and Penn Community Services
Wireless Authentication & Support
NetReg
DNSSEC
Vulnerability Scanning
Security Tools, Education, and Response
PennKey School Support
PGP Whole Disk Encryption LSP Support
XpressConnect
Enhanced AirPennNet Guest Services (FY’13)
Intrusion Detection System
SafeDNS (FY15)
6
6
University of Pennsylvania - Network Planning Task Force
Today’s Agenda
MAGPI Update
Next Generation PennNet
• Core and border routers, building routers
Science DMZ status
Update from the Network Architecture & Security initiative
IP address utilization and IP Address Management (IPAM)
Core server infrastructure
• DNS, DHCP, NTP, Kerberos, RADIUS and SafeDNS
Proposed wired port policy
Networking for the Penn Wharton China Center
Open Discussion
7 University of Pennsylvania - Network Planning Task Force
MAGPI Update
MAGPI is the Penn-sponsored and operated Regional Optical Network for PA, NJ, DE.
MAGPI was founded in 1997 with 2 members; reached a high point of 500 supported R&E institutions through 37 connections in 2011.
Consolidated in early 2014, MAGPI now supports 8 large pipe connections with a strong focus on “big data” research.
8 University of Pennsylvania - Network Planning Task Force
What is MAGPI?
MAGPI Update
The Pennsylvania Research and Education Network (PennREN) is now independently offering Internet2 access.
MAGPI’s focus is now primarily to serve the largest Higher-Ed members within our region, transitioning smaller members to PennREN.
Negotiations for 3 year commitments with 6 major regional universities are in late stages.
MAGPI and Penn have established a new 100Gb connection to Internet2.
9 University of Pennsylvania - Network Planning Task Force
Changes for MAGPI
Next Generation PennNet (NGP)
NGP is an ongoing project to significantly improve performance and reliability of PennNet and to meet the network goals of major campus applications. The primary components are:
• Wiring closet switches
• Building backbones and building routers
• Central wireless infrastructure
• Campus core routers
• Campus border routers
• Core server infrastructure
• Fiber links and Wave Division Multiplexing to key locations in the metro region
• High capacity links to the Internet and Internet2
• Science DMZ
University of Pennsylvania - Network Planning Task Force 10
PennNet Core, Border, and Science
DMZ Topology
University of Pennsylvania - Network Planning Task Force 11
NGP Accomplishments for FY 2014
Completed deployment of new core and border routers.
• Provided 10x increase on core bandwidth to 100Gb, and 100Gb connectivity to Internet2
Provided 10Gb connectivity to most building entrance routers and closet switches.
Completed a major upgrade of wireless controller infrastructure.
Continued progress on closet switch upgrades.
Replaced remaining older building routers to allow for higher density 10Gb connections.
University of Pennsylvania - Network Planning Task Force 12
Building Entrance (BE) Routers
Replaced remaining BE routers (especially for larger buildings) this summer.
New BE routers have greater 10Gb port capacity and allow for closet switch upgrades in larger buildings.
Replacing data center routers with newer larger capacity routers for better performance, higher density, additional features.
University of Pennsylvania - Network Planning Task Force 13
NGP Progress
Switches:
• 1,158 of 1,916 closet switches replaced with 10Gb capable closet switches.
• 151 of 239 buildings are completed. We are currently working in another 28 buildings.
• Most remaining locations are residential and remote buildings off the main campus.
10Gb Connections to Campus Buildings:
• 40 uplink connections are yet to be upgraded.
• Some low volume locations will remain at 1Gb.
University of Pennsylvania - Network Planning Task Force 14
NGP Goals for FY 2015
Complete 10Gb upgrades to remaining BE locations.
Complete data center router upgrades.
Upgrade standard PennNet connections to 1000BaseT.
Continue closet switch deployment.
Continue migration of multimode fiber circuits to single mode fiber circuits.
Continue deployment of the Science DMZ.
Deploy central NAT infrastructure at network border.
University of Pennsylvania - Network Planning Task Force 15
Science DMZ
In April we reported that Penn was successful in getting a Campus Cyberinfrastructure award from the National Science Foundation.
This award was primarily for bringing high bandwidth to campus researchers and establishing a Science DMZ.
A Science DMZ is “a portion of the network designed to optimize for high-performance scientific applications rather than for general purpose business systems or enterprise computing.” *
* Definition from ESNet: https://fasterdata.es.net/science-dmz/
16 University of Pennsylvania - Network Planning Task Force
Science DMZ Progress at Penn
Campus 100 Gbps connection now operational.
Dedicated 100 Gbps Science DMZ switches and circuits deployed.
Initial connections provisioned or underway.
• Connection to South Bank for Dr. Srolovitz (the PI on the Penn grant proposal) is underway.
• Other interfaces soon to be allocated to researchers who wrote support letters.
Deployment of measurement infrastructure and OpenFlow test lab equipment is in progress.
Please see the diagram in subsequent slide.
17 University of Pennsylvania - Network Planning Task Force
PennNet Core, Border, and Science
DMZ Topology
University of Pennsylvania - Network Planning Task Force 18
Network Architecture And Security
Working Group (NASWG)
Last year at NPTF, we introduced the idea of a new group to take a fresh look at PennNet design, particularly in the areas of segmenting and protecting networks.
Since then, a large collaborative team has formed.
• Participation from many schools and centers, sharing local network designs, challenges, solutions
• Participants joined forces to look at new approaches to solving common problems in:
– network segmentation and extension,
– network filtering,
– network access management,
– secure remote access, and
– network visibility.
19 University of Pennsylvania - Network Planning Task Force
Network Architecture And Security
Working Group (NASWG)
Current methods of achieving network segmentation:
• Routed Virtual LAN (VLAN) segments within a building
• Dedicated fiber circuits between buildings
• Routed private VLANs protected by Access Control Lists (ACLs) on the routers
• Site-to-Site infrastructure Virtual Private Networks (VPNs)
Current methods do not scale in a cost-effective way, nor do they meet all segmentation and reach use cases.
Virtualization and overlay technologies would enable new approaches to network segmentation and extension.
20 University of Pennsylvania - Network Planning Task Force
Network Architecture And Security
Working Group (NASWG)
21 University of Pennsylvania - Network Planning Task Force
In combination with the goals of the IRC, and leveraging what is being built for the Science DMZ, NASWG can bring new opportunities for PennNet architecture and security design.
Network Architecture And Security
Working Group (NASWG)
In combination with the goals of the IRC, and leveraging what is being built for the Science DMZ, NASWG can bring new opportunities for PennNet architecture and security design.
22 University of Pennsylvania - Network Planning Task Force
IP Address Management (IPAM) Strategy
PennNet address use growing by 30% annually, largely due to increases in the numbers of wireless devices on campus. Without additional action, we would run out of IPv4 addresses in the next few years.
Strategies include:
Use of Network Address Translation (NAT) in appropriate situations. Possibilities include AirPennNet, AirPennNet-Guest, AirPennNet-Device, Residential Networks.
Use of RFC1918 for internally routed applications.
Continue deployment of IPv6 infrastructure across campus.
Consider outsourcing AirPennNet-Guest (17,868 IPs currently allocated), relying upon ISP address space.
23 University of Pennsylvania - Network Planning Task Force
IPv4 Conservation Efforts
Recent progress has delayed IPv4 exhaustion by 1 year (to Fall 2017).
• Completed conversion of IP addresses on all PennNet Phone networks (recovered 19,000 IP Addresses).
• Bundled wireless AP groups to efficiently share IP addresses.
Preparing to implement NAT services on PennNet.
• NAT purchase and design is complete
• Initial deployment by end of Fall 2014
• Begin conversion of targeted locations, start of CY2015
• Potential gains from NAT: 32,000+ addresses
24
NAT Traffic Flow
ISC Branding, Marketing & Website Project 25
IPv4 Utilization Year Over Year
26
0 10000 20000 30000 40000 50000 60000 70000 80000 90000
AirPennNet
AirPennNet-Device
AirPennNet-Guest
Central Services
Dark
GreekNet
Management/Infrastructure
PennNet (Standard)
PennNetPhone
Private Networks
Resnet
44390
2097
17167
1440
14199
1836
10093
80118
18232
12599
11572
47801
567
17301
1440
30057
1836
9241
80858
0
13055
11572
Sep-14
Oct-13
Core Server Infrastructure
Recent refresh of campus DNS, DHCP, NTP, Kerberos and RADIUS servers.
Common features
• Higher capacity, smaller/cheaper hardware
• Modern software versions
• Substantially increased throughput
• Simplified administration
DNS [Deployed August 2013]
• Anycast IP addresses
• Smaller fault zones, increased availability
DHCP [Deployed July 2013]
• Simplified redundancy model
• Peak sustained request rate of 114/second or 50% capacity
• Exploring static DHCP possibilities
27 University of Pennsylvania - Network Planning Task Force
Core Server Infrastructure
Recent refresh of campus DNS, DHCP, NTP, Kerberos and RADIUS servers.
Common features
• Higher capacity, smaller/cheaper hardware
• Modern software versions
• Substantially increased throughput
• Simplified administration
NTP [Deployed January 2014]
• On-board reference clocks (GPS, AM)
Kerberos KDCs [Deployed June 2014]
• Newer Kerberos code, plus removal of local patches
• Enables high-availability administration
28 University of Pennsylvania - Network Planning Task Force
Core Server Infrastructure
Recent refresh of campus DNS, DHCP, NTP, Kerberos and RADIUS servers.
Common features
• Higher capacity, smaller/cheaper hardware
• Modern software versions
• Substantially increased throughput
• Simplified administration
RADIUS [Deployed August 2014]
• Peak authentication rates up 31% this year to 113/second (sustained for five minutes)
• New service running at roughly 30% of capacity (from 93%)
• New design eliminates risk of cascading failures
• Changes also enables next stage projects (EduRoam, IoT)
29 University of Pennsylvania - Network Planning Task Force
Anycast
virtual
servers
Physical
servers
Primary Campus DNS Resolvers
30 University of Pennsylvania - Network Planning Task Force
Anycast DNS
Anycast
virtual
servers
Physical
servers
SafeDNS Resolvers
31 University of Pennsylvania - Network Planning Task Force
Anycast DNS
r dns1a
r dns1
128. 91. 18. 1
r dns1asdns2a
sdns2
128. 91. 49. 2
sdns2asdns1a
sdns1
128. 91. 18. 2
sdns1a
Primary Secondary Tertiary
PennNet SafeDNS client
The Road to SafeDNS
32 University of Pennsylvania - Network Planning Task Force
Initial pilot128. 91. 19. 240
128. 91. 19. 241
Enhanced pilot128. 91. 18. 2
128. 91. 49. 2
Productionservice
128. 91. 18. 2
128. 91. 49. 2
New users
Front end
Back end
CY14 CY15
Launch
SeptemberNPTF
Evaluate cloud-based service
Adapt proven UPenn solutions
Finalize deployment
OctoberSUG
Campusannouncement
Decisionpoint
Next Steps for DNS/NTP
As SafeDNS moves to production ISC will continue to work with community to migrate to:
• Either RDNS or SDNS service
• High-performance NTP time sources
Proposed Goal: Retire old NOC1, NOC2, NOC3 servers and recover their IP addresses.
• Temporarily maintain legacy DNS addresses as cache-only servers
• Work with community on a schedule to retire old server addresses
33 University of Pennsylvania - Network Planning Task Force
Proposed Wired Port Policy
Purpose is to identify and remove inactive wired ports leading to reduced infrastructure and billing costs.
Initially proposed at July NPTF.
Small working team met in August to discuss policy development.
Proposal details:
• Inactive wired ports disabled after 45 days.
• Additional waiting period of 15 to 30 days, with reactivation if “dead port” ticket is reported.
• Disconnect process follows if no tickets reported.
• Reports 2x/year (Fall/Spring to avoid winter and summer breaks).
34 University of Pennsylvania - Network Planning Task Force
Proposed Wired Port Policy
Next steps
• Seek approval to proceed from NPTF.
• Reconvene the working group.
• Develop policy and fully document process and risks vs. rewards.
• Take the resulting policy through the Network Policy Committee process.
• Present to IT Roundtable for final recommendation and adoption.
35 University of Pennsylvania - Network Planning Task Force
Penn Wharton China Center
Global Engagement Initiative to Extend Penn Presence in Beijing, China
Penn will have classroom presence in World Financial Center Building 16th Floor.
Major technology collaborators are FRES, ISC, and Wharton IT.
ISC assisting Wharton IT staff and the Penn project team with technical design and security consulting for wireless, Telepresence, and network connectivity from China to Penn’s main campus.
PWCC site will have connection to the local Internet and a dedicated Ethernet private line connection to Philadelphia.
• Ethernet for telepresence and Penn traffic for faculty, staff and students visiting PWCC
Work is in progress on selection of IT Integrators and telecommunication service providers.
Targeted turn up and move in date is January 2015.
36 University of Pennsylvania - Network Planning Task Force
Open Discussion
Questions, clarifications, concerns on any topic covered, or additional
topics to be researched.
37 University of Pennsylvania - Network Planning Task Force
Next Meeting
October 13th
Currently planned topics • Identity and Access Management update
• AirPennNet and AirPennNet-Guest updates
• EduRoam
• Wireless for Internet of Things devices
• Cellular and DAS
• Penn+Box
• Project ButtonUP
• NextGen WWW
38 University of Pennsylvania - Network Planning Task Force