StarLight, the Facility

Joe Mambretti, Tom DeFanti, Maxine Brown

Alan Verlo, Linda Winkler

StarLight: A 1 Gigabit and 10 Gigabit Exchange

Abbott Hall, Northwestern University’sChicago downtown campus

StarLight hosts electronic switching and routing for United States national and international Research and Education networks

Since November 2003, StarLight has been optically switching wavelengths between Chicago and Amsterdam

StarLight History

NSF-funded support of STAR TAP (1997-2000) and STAR TAP2/ StarLight (2000-2005), and the High Performance International Internet Services program (Euro-Link, TransPAC, MIRnet and AMPATH).

StarLight, the Facility

• *Flexible* facility for scientists by scientists; input from:– E-science and C-science researchers– Technical leaders of NSF cyberinfrastructure– Academic national and international R&E networks– Next-generation Federal networks– Metro/regional efforts

• A GE and 10 GE exchange for R&E Networks• A MEMS-switched Optical Research Network Exchange• A specialized co-location space for new implementations

and integrations of infrastructure• Home to fiber and circuits from SBC, Qwest, AT&T, Global

Crossing, Level3, T-Systems, Looking Glass, RCN, and I-WIRE, custom integrators, like MCI for DREN

StarLight Facility-ConnectedUS and International Networks

US National• Abilene/HOPI• DREN (DOD)• ESnet (DOE) • NLR• NASA GSFC• NISN (NASA)• NREN (NASA)• UltraScience Net (DOE)

US Other• CAVEwave• FermiWave• I-Light (Indiana)• I-WIRE (Illinois)• LONI (Louisiana) via NLR• MiLR (Michigan LambdaRail)• MREN (Midwest)

• NIH• OMNInet2• TeraGrid• TRECC• UltraLight (HEP) via NLR• USGS• WiscWave (Wisconsin)

• ASnet (Taiwan)• CA*net4 (Canada) • CERNET (China)• GLORIAD-CSTNet

(China) via CA*net4• GLORIAD-Russia via

CA*net4

• GLORIAD/KREONet2 (Korea) via CA*net4

• HARNET (Hong Kong)• JGN-II (Japan)• SINET (Japan)• SURFnet (Netherlands),

which also carries NORDUnet and CESNET

• TaiwanLight (Taiwan) via CA*net4

• TransLight/StarLight IRNC

• UKLight (UK)• US LHCnet (CERN)

E-Science and C-Science Experiments in StarLight Co-Lo Space

• OMNInet2 – Joe Mambretti, Nortel/SBC

• DARPA DWDM-RAM – Joe Mambretti and Nortel Networks

• NSF/DOE Caltech/CERN Cluster for Data Grid development (GriPhyN, PPDG, iVDGL, EU DataGrid) – Harvey Newman

• NSF Distributed Optical Testbed (DOT) – Joe Mambretti and Valerie Taylor

• NSF Logistical Networking – Micah Beck and Yotta Yotta

• NSF National Center for Data Mining Cluster – Bob Grossman

• NSF OptIPuter visualization cluster – Tom DeFanti and Jason Leigh

• NSF MEMS Optical Switching – Tom DeFanti and Jason Leigh

• University of Amsterdam Itanium for Optical switching; Generic AAA – Cees de Laat

• USGS map storage

StarLight Provided Major iGrid 2005 Support

OptIPuter: Major NSF-funded Research Initiative Utilizing StarLight Capabilities

The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences

• An NSF-funded award that focuses on developing technology to enable the real-time collaboration and visualization of very-large time-varying volumetric datasets for the earth sciences and the biosciences

• OptIPuter is examining a new model of computing whereby ultra-high-speed networks form the backplane of a global computer

NIH Biomedical Informatics NSF EarthScope and ORION

http://ncmir.ucsd.edu/gallery.htmlsiovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

Research Network

OptIPuter is a Global Virtual Computer

• Hardware: clusters of computers that act as giant storage, compute or visualization peripherals, in which each node of each cluster is attached at 1 or 10GigE to a backplane of ultra-high-speed networks

• Software: Advanced middleware and application toolkits are being developed for lightpath management, data management and mining, visualization, and collaboration

Commodity GigE Switch

Fibers or Lambdas

The Scalable Adaptive Graphics EnvironmentTo Manage Visual Content on Scalable Tiled Displays

SAGE runs on LambdaVision, a display-rich collaborative work environment (11x5 LCDs)

Moving Data Over these Networks:Ultra-High-Speed Transport Protocols

• Reliable Blast UDP designed for ultra-high-speed bulk data delivery applications• Developed originally in 2000• Achieved 18Gb/s out of 20Gb/s available bandwidth on TeraGrid in

2003.

• LambdaStream designed for ultra-high-speed streaming applications that need high throughput but low latency and low jitter (such as streaming graphics)• Developed in 2003/04• LambdaStream achieves this by adaptive rate adjustment AND

attempting to predict the CAUSE of loss using techniques from Mobile Adhoc Networks (MANETs).

EVL

10GE OptIPuter CAVEwaveon the National LambdaRail

Source: Tom DeFanti, OptIPuter co-PI

Calient DiamondWave Switches at StarLight and NetherLight

• 128x128 Calient O-O-O 3D MEMS switch at StarLight

• 64x64 Calient switch at NetherLight

• 1% the cost of routing

Photonic Interdomain Negotiator (PIN)Application-Centric Scheduling of Photonic Networks

VisualizationEVL (Chicago)

Cluster

OC-192

PIN

Cluster

Calient Photonic Switch

Cluster Cluster

Calient Photonic Switch

GlimmerglassPhotonic Switch

Correlation / FilteringStarLight (Chicago)

Data Access UvA (Amsterdam)

Muxed & DeMuxed DWDM

PIN PIN

PPBAC PDCPDC

Joe Mambretti (NU), Eric He, Cees de Laat (UvA), Oliver Yu (UIC)

Bring Us Your Lambdas

www.startap.net/starlight

Thanks to our Sponsors and Collaborators

• StarLight planning, research, collaborations, and outreach efforts have been made possible, in major part, by funding from: – US National Science Foundation (NSF) awards SCI-9980480, SCI-

9730202, CNS-9802090, CNS-9871058, SCI-0225642, and CNS-0115809

– State of Illinois I-WIRE Program, and major UIC cost sharing– Northwestern University for facility space, engineering and

management• US NSF/CISE and US DoE/Argonne National Laboratory for StarLight

and I-WIRE network engineering and design• Bill St. Arnaud of CANARIE and Kees Neggers of SURFnet for

networking leadership• Larry Smarr of Calit2 for I-WIRE and OptIPuter leadership