Joint Techs / APAN

30
Joint Techs / APAN Joint Techs / APAN Honolulu Honolulu Mark Johnson Mark Johnson MCNC (NCREN, NCNI, NCLR, MCNC (NCREN, NCNI, NCLR, …) …) [email protected] [email protected]

description

Joint Techs / APAN. Honolulu Mark Johnson MCNC (NCREN, NCNI, NCLR, …) [email protected]. How can an R&E network afford to build an advanced network?. Use the obvious strategy of obtaining donations from providers and equipment vendors and the use of grants - PowerPoint PPT Presentation

Transcript of Joint Techs / APAN

Page 1: Joint Techs / APAN

Joint Techs / APANJoint Techs / APANHonoluluHonolulu

Mark JohnsonMark Johnson

MCNC (NCREN, NCNI, NCLR, …)MCNC (NCREN, NCNI, NCLR, …)

[email protected]@ncren.net

Page 2: Joint Techs / APAN

How can an R&E network afford to How can an R&E network afford to build an advanced network?build an advanced network?

Use the obvious strategy of obtaining Use the obvious strategy of obtaining donations from providers and equipment donations from providers and equipment vendors and the use of grantsvendors and the use of grants

Make more efficient use of the available Make more efficient use of the available (scarce) resources - MORPHnet(scarce) resources - MORPHnet

Page 3: Joint Techs / APAN

Infrastructure Use ExamplesResearch based on measurements of real user Internet traffic (and not just univ-to-univ traffic) and visibility into Internet BGP for the first time since NSFnet Production use for higher ed and K-12 AUP-free commodity Internet access and inter-GigaPoP transit backupResearch needing its own L2 links with the capability to do complex topologies but where speed is not the primary focus and 1GE or lower ports are sufficient, e.g., multicast routingProduction use for cases where shared IP service is not acceptable but also dedicated 10G waves not needed either, e.g., remote instrument controlResearch needing its own L1 links and/or dedicated 10G bandwidth, e.g., very large MTU performance, XTP implementationProduction use of dedicated (multiple) 10G bandwidth, e.g., DTF/ETF cluster supercomputers "backplane" interconnect, federal agency mission use, international connections transitResearch needing its own dark fiber full spectrum and/or deployment of breakable L1 gear, e.g., optical packet switching, IP-optics unified control plane, 100GE optics

Experimental or breakable

L3 gear

Production L3 gear

Production fiber (2nd pair)

NLR operated

Research use

Infrastructure

Production fiber (1st pair)

Experimental or breakable

L1-3 gear

Production point-to-point wave service (Cisco COTS DWDM gear) - 10GE, 1GE, OC192 and OC48 waves

Production IP service (Cisco COTS routers) - 10GE and

1GE ports

Experimental or breakable

L2-3 gear

Production L2-3 gearProduction Ethernet service

(Cisco COTS switches) - 1GE ports

NLR or its production customer or researcher operated

Production use

Production and experimental infrastructure(MORPHnet concept) and their use

Page 4: Joint Techs / APAN

What does a user want from What does a user want from an optical network?an optical network?

An end-to-end path (lightpath) where the An end-to-end path (lightpath) where the endpoints are not defined by the limits of a endpoints are not defined by the limits of a single carrier’s networksingle carrier’s network

Page 5: Joint Techs / APAN

LightpathsLightpaths

a lightpath is defined to be a a lightpath is defined to be a fixed bandwidth connection fixed bandwidth connection between two network elements, between two network elements, such as IP routers or ATM such as IP routers or ATM switches, established via the switches, established via the optical networkoptical network Ietf draft on lightpath attributesIetf draft on lightpath attributes

Page 6: Joint Techs / APAN

Lightpath attributesLightpath attributes

It is assumed that a lightpath will have a It is assumed that a lightpath will have a number of attributes that describe it such number of attributes that describe it such as framing, bandwidth, etcas framing, bandwidth, etc

Canarie asserts that across a given AS a Canarie asserts that across a given AS a lightpath may be abstracted to look like a lightpath may be abstracted to look like a single (possibly blocking) cross-connect single (possibly blocking) cross-connect switch interface.switch interface.

Page 7: Joint Techs / APAN

working examples of working examples of LightpathsLightpaths

All Optical wavelength on WDM systemAll Optical wavelength on WDM system SONET channelSONET channel Point to point ethernetPoint to point ethernet ATM CBR circuitATM CBR circuit MPLS LSR with defined QoSMPLS LSR with defined QoS FiberchannelFiberchannel SMPTE 259SMPTE 259 G.709 (Digital Wrapper)G.709 (Digital Wrapper)

Page 8: Joint Techs / APAN

ProblemsProblems

Intra-domainIntra-domain Provisioning of network capacity across Provisioning of network capacity across

network elements within an ASnetwork elements within an AS O&MO&M

Inter-domainInter-domain Provisioning of network capacity across Provisioning of network capacity across

multiple AS’smultiple AS’s O&MO&M

In this environment the user has to handle In this environment the user has to handle performance and fault managementperformance and fault management

Page 9: Joint Techs / APAN

LightpathLightpath

Carrier A Carrier BCarrier C

User desires red path but must negotiate and manage provisioning of green, orange, and blue paths

Page 10: Joint Techs / APAN

ApproachesApproaches

Methods of defining, provisioning, and Methods of defining, provisioning, and modifying existing services within a modifying existing services within a management domainmanagement domain G.ASTNG.ASTN GMPLSGMPLS

Methods of linking paths from multiple domainsMethods of linking paths from multiple domains UCLPUCLP

Non-traditional techniques for provisioning Non-traditional techniques for provisioning capacity between endpointscapacity between endpoints OBS/JITOBS/JIT

Page 11: Joint Techs / APAN

GMPLSGMPLS

Generalized MPLS signaling to identify the Generalized MPLS signaling to identify the following path types:following path types: Traditionally statistically multiplexed labeled paths Traditionally statistically multiplexed labeled paths

such as ATM or Ethernetsuch as ATM or Ethernet Time division multiplexed paths such as SONET Time division multiplexed paths such as SONET

where timeslots are the labelwhere timeslots are the label Frequency division multiplexed services such as Frequency division multiplexed services such as

wavelengths where frequency is the labelwavelengths where frequency is the label Space division multiplexed services such as fibers in Space division multiplexed services such as fibers in

a bundle where position in the real world is the labela bundle where position in the real world is the label

Page 12: Joint Techs / APAN

Division of laborDivision of labor

Control planeControl plane Signaling, routing, Protection /restorationSignaling, routing, Protection /restoration

TransportTransport Adaptation, Aggregation, Discovery,data Adaptation, Aggregation, Discovery,data

integrity, transmissionintegrity, transmission ManagementManagement

Management of Faults, Management of Faults, configuration/provisioning, accounting, configuration/provisioning, accounting, performance measurement, securityperformance measurement, security

Page 13: Joint Techs / APAN

Division of laborDivision of labor

Network Topology Map

Topology Policy Constraints

Label ForwardingInformation base

Control plane

based on IP Routing

IP

Today Future

Forwarding plane

Optical

ATM

Drawing poorly copied from Cisco Systems

Page 14: Joint Techs / APAN

GMPLS Protocol DiagramGMPLS Protocol Diagram

FIBER

SONET WavelengthSwitching

MAC/GE ATM FrameRelay

Adaptation LayerIP

LMP RSVP-TE CR-LDP-TE

TCP

OSPF-TEBGP

UDP

Page 15: Joint Techs / APAN

UCLPUCLP

Canarie is developing a system including Canarie is developing a system including protocols and directories and registration protocols and directories and registration mechanisms Addresses interdomain mechanisms Addresses interdomain issues:issues:

Registration of available path componentsRegistration of available path components Directory service for those componentsDirectory service for those components Provisioning of end to end path which Provisioning of end to end path which

could use intra-domain tools such as could use intra-domain tools such as GMPLSGMPLS

Page 16: Joint Techs / APAN

JIT/OBS view of Optical Network DilemmaJIT/OBS view of Optical Network Dilemma

Goal: Lower cost by:Goal: Lower cost by: Minimizing OEOMinimizing OEO Creating larger transparency islandsCreating larger transparency islands

But:But: Dedicated Dedicated is overkill (expensive) is overkill (expensive) Low speed apps. need fine grain mux capabilityLow speed apps. need fine grain mux capability

And:And: Existing fine grain multiplexing today requires Existing fine grain multiplexing today requires

electronics hence OEO conversionelectronics hence OEO conversion

Page 17: Joint Techs / APAN

Technology gapTechnology gap

Requires optical buffersRequires optical buffers Immature, expensive, low densityImmature, expensive, low density Buffers in net lead to complexityBuffers in net lead to complexity

IP is a COMPLEX protocolIP is a COMPLEX protocol Hardware implementations only recentlyHardware implementations only recently Creates cost and technology barrierCreates cost and technology barrier

Gauger et al., “Determining offset times in optical burst switching networks”, COST 266, Zagreb, June 2001.

Page 18: Joint Techs / APAN

Optical Cell SwitchingOptical Cell Switching

TDM dWDMTDM dWDM All wavelengths on fiber switched togetherAll wavelengths on fiber switched together

Pluses and minusesPluses and minuses Simpler core networkSimpler core network Need chromatic time correctionNeed chromatic time correction Requires frame synchronizationRequires frame synchronization Low utilization of wavelenghtsLow utilization of wavelenghts

Lucent is major proponentLucent is major proponent

Page 19: Joint Techs / APAN

Three Competing IdeasThree Competing IdeasNeed for Need for optical optical bufferingbuffering

Synchro-Synchro-nizationnization

Relative Relative timeline to timeline to commercial commercial viabilityviability

Relative Relative complexitycomplexity

commentscomments

OPSOPS criticalcritical NoNo Not in our Not in our career lifetimecareer lifetime

Complex Complex switching and switching and protocolprotocol

Requires Requires optical logicoptical logic

OCSOCS nono yesyes 5 to 10 years 5 to 10 years Simplest Simplest switch core, switch core, with most with most complex line complex line cardscards

Requires Requires chromatic chromatic and frame and frame allignment, allignment, low low utilizationutilization

OBSOBS No longer No longer seen as seen as necessary necessary or desirableor desirable

Not Not required required for JIT, for JIT, limited limited sync for sync for JET JET

2 years with 20 2 years with 20 ms, 7 years ms, 7 years with 10 ns with 10 ns reconfigurationreconfiguration

Simple line Simple line with cards with cards modestly modestly complex complex switch coreswitch core

Requires Requires demultiplexindemultiplexing and g and conversionconversion

Page 20: Joint Techs / APAN

JIT Fundamental ValuesJIT Fundamental Values

Low latency is first priorityLow latency is first priority Tell and go vs. tell and waitTell and go vs. tell and wait May sacrifice link utilizationMay sacrifice link utilization

JET and HorizonJET and Horizon

Aggressive protocol simplificationAggressive protocol simplification A pox on buffers (optical delay lines)A pox on buffers (optical delay lines)

Leads to un-necessary protocol and switch complexitiesLeads to un-necessary protocol and switch complexities Leads to greater link speed and lower latencyLeads to greater link speed and lower latency

Keep data in opticsKeep data in opticsNo legacy assumptionsNo legacy assumptionsResult: high throughput, min. latency and jitterResult: high throughput, min. latency and jitter

Page 21: Joint Techs / APAN

JIT - OBS ApproachJIT - OBS Approach

Switched light path network Switched light path network Large all-optical islandLarge all-optical island No buffers in data channelNo buffers in data channel

Avoids immature device technologyAvoids immature device technology No buffer overflow in network No buffer overflow in network

Data and signaling channel isolationData and signaling channel isolation Single out of band signaling channel per fiberSingle out of band signaling channel per fiber Signaling msgs. undergo OEO, processed by Signaling msgs. undergo OEO, processed by

intermediate nodes intermediate nodes Network intelligence is concentrated at edgeNetwork intelligence is concentrated at edge

SIMPLE protocol implemented in hardwareSIMPLE protocol implemented in hardware

Page 22: Joint Techs / APAN

ECOnetECOnet

Create a confederation of fiber linked NRT projects:Create a confederation of fiber linked NRT projects: BOSnet: BOSnet:

MIT Lincoln LabsMIT Lincoln Labs dark fiber Boston to Washington DCdark fiber Boston to Washington DC

ATDNet: ATDNet: Naval Research Lab (& others) Naval Research Lab (& others) dark fiber within the Washington D.C. metro areadark fiber within the Washington D.C. metro area

ECO-South (proposed):ECO-South (proposed): MAX/MCNC/SOXMAX/MCNC/SOX Dark fiber from Washington to Research Triangle Park and then Dark fiber from Washington to Research Triangle Park and then

to Atlantato Atlanta

Page 23: Joint Techs / APAN

East Coast Optical NetworkEast Coast Optical NetworkECOnetECOnet

MAX/ATDNet Washington, DC

MCNC/NCREN Raleigh, NC

GaTech/SOX Atlanta, GA

MITLL Boston, MA

BOSnet

ECO-South

Page 24: Joint Techs / APAN

Illustrates need to evaluate Entire systemIllustrates need to evaluate Entire system

Fiber, amps, DCUs, maintenance and Fiber, amps, DCUs, maintenance and Rent can become dominant costsRent can become dominant costs

Page 25: Joint Techs / APAN

Two fiber routesTwo fiber routes

Route A Miles 3R's ILA's

Washington, D.C. to Richmond 129 - 1 Richmond to Raleigh 170 - 2 Raleigh to Charlotte 208 - 3 Charlotte to Atlanta 260 - 3

TOTAL 767 9 Route B Miles 3R's ILA's

Washington, D.C. to Raleigh 476 - 9 - Raleigh to Atlanta 563 - 12 -

TOTAL 1,039 21

Page 26: Joint Techs / APAN

Fiber costFiber cost

Number of Fibers

Fiber mile Price

20 YR IRU Price fiber totals

2 500$ 129,000$

2 500$ 170,000$ 299,000$

2 500$ 208,000$

2 500$ 260,000$ 468,000$

767,000$ 767,000$

2 -$ -$ -$

2 -$ -$ -$

- - -

Route A

Washington, D.C. to RichmondRichmond to RaleighRaleigh to CharlotteCharlotte to Atlanta

TOTAL Route B

Washington, D.C. to Raleigh Raleigh to Atlanta

TOTAL

Page 27: Joint Techs / APAN

Amps, colo, maintenanceAmps, colo, maintenance

Route A

Washington, D.C. to RichmondRichmond to RaleighRaleigh to CharlotteCharlotte to Atlanta

TOTAL Route B

Washington, D.C. to Raleigh Raleigh to Atlanta

TOTAL

ILA Racks (20ADC) ila rent

POP Racks

(20ADC) POP rent maint amp HW amps

600$ 21,600$ 800 9600 29,900$ 126,000$ 504,000$

600$ 43,200$ 800 9600 46,800$ 126,000$ 882,000$

64,800$ 76,700$ 1,386,000$

400$ 43,200$ 800$ 9600 0 126,000$ 1,260,000$

400$ 57,600$ 800 9600 0 126,000$ 1,638,000$

100,800 2,898,000

Page 28: Joint Techs / APAN

5 year total5 year total

Route A

Washington, D.C. to RichmondRichmond to RaleighRaleigh to CharlotteCharlotte to Atlanta

TOTAL Route B

Washington, D.C. to Raleigh Raleigh to Atlanta

TOTAL

total year 1 (dim)

total year 2 (dim)

total year 3 (dim)

total year 4 (dim)

total year 5 (dim) total yr 1-5

624,900 120,900$ 120,900$ 120,900$ 120,900$ 1,108,500

1,075,200 183,600$ 183,600$ 183,600$ 183,600$ 1,809,600

1,700,100 304,500 304,500 304,500 304,500 2,918,100

1,312,800 43,200$ 43,200$ 43,200$ 43,200$ 1,485,600

1,705,200 57,600$ 57,600$ 57,600$ 57,600$ 1,935,600

3,018,000 100,800 100,800 100,800 100,800 3,421,200

Page 29: Joint Techs / APAN
Page 30: Joint Techs / APAN

NCNI WDM NetworkNCNI WDM Network

Cisco RTP MCNC

UNC

Duke

NCSU RLGH

DCU

DCU

DCU

DCU

DCU

DCU

DCU

MCNC

Engineering Notes:•Ring Circumference = 178.1099km•SMF-28 Fiber

EDFA amplifier

DispersionCompensation Unit

SMJ 5-27-03

15454 Node

NetworkManagementAccess