Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project...
-
Upload
dayana-skillen -
Category
Documents
-
view
226 -
download
0
Transcript of Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project...
![Page 1: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/1.jpg)
“Internet2 Presentation”
Brian Stengel, Director of Operations, Kinber
Peter J. Heverin, Project Manager, KinberMike Carey, Network Engineer, Kinber
Jon Paul Herron, Director of Engineering, GlobalNOCLuke Fowler, Senior Manager, Systems Engineering, GlobalNOC
Marianne Chitwood, Director of Operations, GlobalNOC
September 20th, 2012
![Page 2: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/2.jpg)
What is PennREN
• Capital Budget - $128,958,031– Federal Stimulus Funds - $99,660,678– Matching Funds - $29,297,353
• Outside Plant Infrastructure Constructed for PennREN – 48 Stands of NZD Fiber optic Cable– 1700± Route Miles– Outsourced Fiber Maintenance
• 13 Optical Regeneration Service Nodes
• 56 Service Distribution Access Nodes
a high-speed, state-wide, research & education serving healthcare, K-20 and the public good
![Page 3: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/3.jpg)
Project Route
Total PennREN Fiber route is estimated at 1,613 miles.
The route consists of the following;
• 1,086 miles of new aerial construction.• 486 miles of aerial overlash.• 14 miles of new underground construction.• 27 miles of leased underground conduit.
Network Backbone Engineering is 95% complete overall.
3
![Page 4: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/4.jpg)
Project Route
4
![Page 5: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/5.jpg)
PennREN LocationsService Nodes
Lehigh University
401 N. Broad
Penn State Hershey Medical Center
Indiana University of Pennsylvania
Allegheny Center Mall
Slippery Rock University of PennsylvaniaPenn State Erie (Behrend)
Clarion University of Pennsylvania
Penn State Dubois
Penn State – State College
Windstream
Bucknell University
University of Scranton
Amp NodesEast Stroudsburg University of Pa
Kutztown University of Pa
West Chester University of Pa
Millersville University of Pa
Shippensburg University of Pa
Pitt Johnstown
Community College of Beaver Co
Allegheny College
Pitt Titusville
Penn State Hazelton
Access Nodes46 Locations on PennREN fiber
![Page 6: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/6.jpg)
Project Teams
• Hardware procurement• Design• Integration• Project Management• Professional Services
• Plan/Design• Optical engineering• Field services• Professional services
• Staging• Shipping• Configuration• Testing• Turn-up• Field Services
• Plan/Design• Consulting• Engineering• Support• Training
• Fiber OSP• Building entrances• Cabinets, Installation• Splicing• Testing• Last mile solutions
• Earthlink sourcing for OOB MPLS Network
• Last mile solutions
• Service Desk, Tier 1, Tier 2• Systems and Network Tools• Network Engineering• Net, Perf, Change Management• Operations support
![Page 7: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/7.jpg)
Node Installation Status
Segment in Production
Equipment Installed
Under Construction
![Page 8: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/8.jpg)
Co-Location at Member Sites
Construction Diagrams
![Page 9: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/9.jpg)
Optical Core/DWDM System
![Page 10: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/10.jpg)
Packet Core – MPLS
![Page 11: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/11.jpg)
Plan – Build – Integrate
![Page 12: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/12.jpg)
Peering Points (Initial)
Internet
R&E
R&E
Internet
![Page 13: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/13.jpg)
Network Management
External Networks
External Networks
Out-of-band access
In-band management
Performance Measurement Servers at every Service Node
PS1PS2PS3
DB
![Page 14: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/14.jpg)
Operations - SupportGRNOC provides to KINBER:• Service Desk – 24x7x365 call center support, ticket management, technical
support coordination, and workflow support• Network Engineering – Expert network engineers work with the Service Desk
to ensure fast problem resolution, provisioning, and strategic engineering and planning
• Software and Systems – Provides support through a fully integrated system of network management, measurement, and visualization tools
The GlobalNOC at Indiana University provides carrier-grade operations, tools, and network expertise while placing a singular focus on the unique requirements of the research and education (R&E) Community
GRNOC supports 20+ R&E networks across the country
![Page 15: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/15.jpg)
Operations - Maintenance
• Warranty• Hardware replacement
(pre-ship)• Software maintenance
• Warranty• Hardware
replacement (pre-ship)
• Software maintenance
• Fiber Maintenance• Emergency Restoral• Routine
Maintenance• OSP Records
• Host IT/Site teams provide remote eyes and hands support upon request
• KINBER Network Engineers
![Page 16: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/16.jpg)
Foundation for Services
West DWDM Ring
East DWDM Ring
MPLS PE Switches
ASBR Router
Multi-DegreeROADMs
External Networks
External Networks
3rd Party Providers
![Page 17: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/17.jpg)
On-Net: Delivery
Internet
R&E R&E
Internet
AccessNode
AccessNode
AccessNode
AccessNode
Access Nodes
![Page 18: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/18.jpg)
Off-Net: Delivery
Internet
R&E R&E
Internet
MemberOff-Net
First/Last MileCommercial
Co-Lo
CXCarrier
Private
NNI
![Page 19: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/19.jpg)
PennREN
EPC – Ethernet Port Connection Service
Customer Router
Service Node: PE Switch
Customer Subscribes to 1/10GE Ethernet Port(s)
R&E networks
VPLS (pt – pt, multipoint)
KMEX
Commodity Internet
![Page 20: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/20.jpg)
Services• Services available to a customer with an EPC
R&E Transit
KINBER Member Exchange
VPLS – Pt-Pt, Multipoint
Commodity Internet
Member A
Member AMember C
Member B
![Page 21: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/21.jpg)
Services• Community-wide, distributed service for members to
exchange traffic across a common network. Similar to an Internet Exchange
• Member-to-Member peering, ad-hoc R&E activities• Best-effort traffic exchange within the community
R&E Transit
KINBER Member Exchange
VPLS – Virtual Private LAN Service
(Point-to-PointMulti-Point)
Commodity Internet
• Transit service to major R&E networks such as Internet 2, ESNet, NLR…
• Provided by KINBER affiliates
• Virtual private networks with committed bandwidth can be established using VPLS instances in the PennREN network
• VPWS – Virtual Private Wire Service – Point to Point• VPLS – Virtual Private LAN Service - Multipoint
• Access to commodity Internet service is available over the PennREN network
![Page 22: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/22.jpg)
Optical – Wave/Lambda Services
• Optical Waves 10G can be provisioned across the networkMember AMember A
Member AMember B Member B
![Page 23: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/23.jpg)
• PerfSONAR Measurement Archives allow exchange of data with other network operators
• Regularly scheduled testing across the backbone
• User-initiated testing for applications such as problem diagnosis
• Multiple routing tables allow us to support both 1G and 10G testpoints on a single host
Performance Measurement
![Page 24: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/24.jpg)
Performance Measurement
Deploying performance measurement servers at each service node
3 servers per site:– Active throughput measurement (1G and 10G)– Active latency measurement– local data collection / ad hoc performance measurement
Specific performance measurement tools include:
BWCTL
OWAMP
MaDDash
24
![Page 25: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/25.jpg)
Network Measurement
• GlobalNOC tool-set updated to support all PennREN devices, including:– SNAPP – High-resolution SNMP-based network
utilization data– LLAMA – DWDM layer performance measurement
data– Central storage of other passive data like syslog,
configuration, flow, etc. for regular and ad-hoc processing & analysis.
![Page 26: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/26.jpg)
PennREN Service Desk
• GlobalNOC Specialized Support Technician • Footprints PennREN project created – integrated to
TickMon, Operations Calendars, Trouble Tickets• Telephone number for PennREN customers integrated
into shared GlobalNOC phone queue• Email established [email protected] to receive customer
inquiries and or communications from vendors….this is monitored 24x7x365
![Page 27: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/27.jpg)
PennREN Service Desk
• Network and Member impact guidelines defined
• Web form for customers/vendors to submit trouble tickets into Footprints
• Change management process/form created• Internal documentation developed/published
for staff training and reference
![Page 28: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/28.jpg)
PennREN Service Desk
• Pro-active network monitoring• Support for scheduled maintenances and
changes• Vendor coordination• Customer install process• Reporting• Security• Tools and Communications
![Page 29: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/29.jpg)
Weighing network needs
![Page 30: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/30.jpg)
A new network is more than a construction project
• It’s a system:– Goals– Infrastructure– People– Services– Operations– Business
• Make decisions and plans early (also, there will be more decisions than you think, so watch for bottlenecks)
• The work on all of these areas starts right away
![Page 31: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/31.jpg)
Service Definition
• Unrealistic to imagine services will be completely defined from the beginning of the design
• But, enough understanding is needed to guide the design and build plan
• Early “anchor” users help a lot!
![Page 32: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/32.jpg)
Communications
• Communication needs:– to be high-bandwidth– Early is good– Changing staff can be disruptive, do it carefully.– Informal is good– Multi-channel is good– reliable technologies are good– Face to face is good
![Page 33: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/33.jpg)
Documentation
• Documentation:– Have a place for documents– Keep the place for documents clean– Keep the purpose of each document clear/distinct
![Page 34: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/34.jpg)
Operations
• Operations Preparedness:– Start early, there are a LOT of things to think
about, especially:• Turn-up/acceptance process• Expectations for facilities• How and where to keep network data
![Page 35: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/35.jpg)
Build-outs make for lots of information
• Need to be ready to put it somewhere where it’ll be usable later
• Data entry is cheap• Entering/documenting is best when the
information and its context are fresh
![Page 36: Internet2 Presentation Brian Stengel, Director of Operations, Kinber Peter J. Heverin, Project Manager, Kinber Mike Carey, Network Engineer, Kinber Jon.](https://reader035.fdocuments.us/reader035/viewer/2022081504/55189e13550346b31f8b474e/html5/thumbnails/36.jpg)
The lowly management network
• Sometimes doesn’t get enough attention• It can get crazy complicated or crazy expensive• NOTE: in an SDN world, this becomes even
MORE important!