Sponsored by the National Science Foundation GENI as a Virtual Laboratory for Networking and...
-
Upload
norman-riley -
Category
Documents
-
view
212 -
download
0
Transcript of Sponsored by the National Science Foundation GENI as a Virtual Laboratory for Networking and...
Sponsored by the National Science Foundation
GENI as a Virtual Laboratory for Networking and Distributed Systems
ClassesWorkshop 7 at SIGCSE ‘14
Sarah Edwards
GENI Project Office
Sponsored by the National Science Foundation 2GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Agenda
Presentation and Demo: An Introduction to GENI (45min)
Hands-on: Getting Started with GENI (45min)
Break (10min)
Hands-on: IPv4 Routing Assignment (50min)
Resources for Instructors (15min)
Miscellaneous GENI Topics & Wrap-Up (15min)
Sponsored by the National Science Foundation
GENIExploring Networks of the Future
www.geni.net
Sponsored by the National Science Foundation 4GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Outline
• GENI – Exploring future internets at scale• The GENI Concept• Building GENI• Experimental and Classroom use of GENI• What’s next for GENI?• GENI: An experimenter’s view
Sponsored by the National Science Foundation 5GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Credit: MONET Group at UIUC
Society Issues
We increasingly rely on the Internet but are
unsure we can trust its security, privacy or
resilience
Science Issues
We cannot currently understand or predict the
behavior of complex,large-scale networks
Innovation Issues
Substantial barriers toat-scale experimentation with new architectures, services,
and technologies
Global networks are creatingextremely important new challenges
Sponsored by the National Science Foundation 6GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI: Infrastructure for Experimentation
GENI provides compute resources that can be connected in experimenter specified Layer 2 topologies.
Sponsored by the National Science Foundation 7GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI: Infrastructure for Experimentation
GENI provides compute resources that can be connected in experimenter specified Layer 2 topologies.
Sponsored by the National Science Foundation 8GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Multiple GENI Experiments run Concurrently
Infrastructure can be shared between slices
Experiments live in isolated “slices”
Sponsored by the National Science Foundation 9GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI is “Deeply Programmable”
I install software I want throughout my network slice (into routers, switches, …) or control
switches using OpenFlow
Experimenters can set up custom topologies, protocols and switching of flows
Sponsored by the National Science Foundation 10GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI Compute Resources
GENI Racks
Existing Testbeds(e.g. Emulab)
GENI Wireless compute nodes
Sponsored by the National Science Foundation 11GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI Networking Resources
Networking within a Rack
National Research Backbones(e.g. Internet2)
Regional Networks(e.g. CENIC)
WiMAX Base Stations
Sponsored by the National Science Foundation 12GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Outline
• GENI – Exploring future internets at scale• The GENI Concept• Building GENI• Experimental and Classroom use of GENI• GENI: An experimenter’s view
Sponsored by the National Science Foundation 13GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
“I have a great idea.”
“That will never work.”
A bright idea
Sponsored by the National Science Foundation 14GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Let’s try it out!
My new architecture worked great in the lab, so now I’m going to try a larger experiment for a few months.
He uses a modest slice of GENI, sharing its infrastructure with many other concurrent experiments.
Sponsored by the National Science Foundation 15GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
It turns into a really good idea
His slice of GENI keeps growing, but GENI is still running many other concurrent experiments.
This service looks very useful
Sponsored by the National Science Foundation 16GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
“Looks like an app to me.”
“It’s my very own GENI slice.”
Attracts real users
Sponsored by the National Science Foundation 17GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
“Boy did I learn a lot!”
“What a cool service.”(I wonder how it works.)
“I always said it was a great idea.”
(But way too conservative.)
Sponsored by the National Science Foundation 18GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
If you have a great idea, check out theNSF CISE research programs for current opportunities.
??
Sponsored by the National Science Foundation 19GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Moral of this story
GENI is meant to enable . . .– At-scale experiments– Internet-incompatible experiments– Both repeatable and “in the wild”
experiments– ‘Opt in’ for real users– Instrumentation and measurement
tools
GENI creates a huge opportunity for ambitious research!
Sponsored by the National Science Foundation 20GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Outline
• GENI – Exploring future internets at scale• The GENI Concept• Building GENI• Experimental and Classroom use of GENI• GENI: An experimenter’s view
Sponsored by the National Science Foundation 21GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Growing GENI’s footprint
Sponsored by the National Science Foundation 22GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
FederationGENI grows by GENI-enabling heterogeneous infrastructure
Avoid technology “lock in” and grow quickly by incorporating existing infrastructure
Backbone #1
Regional
GENI Rack
GENI Rack
Access#1
CommercialClouds
CorporateGENI suites
Non-USTestbeds
ResearchTestbed
Campus
My experiment runs acrossthe evolving GENI federation.
My GENI Slice
This approach looks remarkably familiar . . .
Sponsored by the National Science Foundation 23GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Federation Extends the Reach of GENI and International Peer Testbeds
Initial plan to federate testbeds on five continents
Sponsored by the National Science Foundation 24GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
“At scale” GENI prototype
Campus photo by Vonbloompasha
Build GENI at sufficient scale
Infeasible to build a testbed as big as the Internet
GENI-enabled campuses,students as early adopters
HP ProCurve 5400 Switch
NEC WiMAX Base Station
GENI-enabledequipment
GENI-enable testbeds, commercial equipment, campuses, regional and backbone networks
Sponsored by the National Science Foundation 25GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI architecture
• Flexible network / cloud research infrastructure
• Also suitable for physics, genomics, other domain science
• Support “hybrid circuit” model plus much more (OpenFlow)
• Distributed cloud (racks) for content caching, acceleration, etc.
MetroResearch
Backbones
InternetISPU N I V E R S I T YU N I V E R S I T Y
U N I V E R S I T YU N I V E R S I T Y
Regional Networks Campus
g
g
gLegend
GENI-enabled hardware
Layer 3Control Plane
Layer 2Data Plane
Sponsored by the National Science Foundation 26GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Creating and deploying GENI racks
GENI RackInstalled at GPO – Feb 22, 2012
Ilia BaldineRENCIMore resources / rack,fewer racks
Rick McGeerFewer resources / rack,
more racks
Sponsored by the National Science Foundation 27GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI WiMAX 2013
• Researcher-owned,• researcher-operated• 4G cellular systems
• 26 Wimax Base Stations in 13 Sites
• Sliced, virtualized
and interconnected
On the AirNot On the Air
Sponsored by the National Science Foundation 28GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI WiMAX Agreements
• Agreement with Clearwire– Clearwire and Rutgers University have signed a master agreement– encompassing all WiMAX sites, to ensure operation in the EBS Band.– An emergency stop procedure, in case of interference with Clearwire
service, has been agreed upon.
• GENI Mobile Virtual Network Operator (MVNO)
- Partner with Sprint and Arterra (a Sprint partner) to create and operate an (MVNO) that serves the academic research community
- The effort is led by Jim Martin, Clemson Univ, and is underway with a 1 year NSF EAGER
Agreements in place to broadcast on our own frequency and to support roaming
Sponsored by the National Science Foundation 29GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
GENI Operations
GMOC: GENI Meta-operation Center• Keeps track of outages• Notification system for resource reservation• Monitors most GENI Aggregates
GMOC Google Calendar keeps track of reservations/outages
Sponsored by the National Science Foundation 30GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Outline
• GENI – Exploring future internets at scale• The GENI Concept• Building GENI• Experimental and Classroom use of GENI• GENI: An experimenter’s view
Sponsored by the National Science Foundation 31GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
How is GENI being Used?
Research• Future Internet
Architectures• Software defined
networking• Large scale evaluation of
protocols
Education• Networking and
Distributed systems classes
• Cloud computing classes• WiMAX classes
As of October 2013, GENI had over 1200 users!
Sponsored by the National Science Foundation 32GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Three FIA Teams have Slices on GENI
GENI is the only testbed that can support these teams.
XIA (demo at GEC15)
NDN (demo at GEC 13)
MobilityFirst (demo at GEC 12 & GEC18)
Sponsored by the National Science Foundation 33GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Virtual Desktop Cloud
Prasad Calyam, University of Missouri, Columbia
Program realtime load-balancing functionality
deep into the network to improve QoE
Sponsored by the National Science Foundation 34GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Spring 2014: Jeannie Albrecht (Williams College) Suman Banerjee (U. of Wisconsin) Deniz Gurkan (U. of Houston) Thanasis Korakis (NYU Poly) Yaoqing Liu (Clarkson U) Shivendra Panwar (NYU Poly) Robert Ricci (Utah) Violet Syrotiuk (Arizona State U) Bing Wang (U. of Conn) KC Wang (Clemson) Vasillis Maglaris (NTUA Greece) Gaia Maselli (Sapienza University of Rome – Italy)
GENI in the Classroom – A great success!
Jeannie Albrecht (Williams College) withstudents from her Spring 2012 DistributedSystems class
Fall 2013: Prasad Calyam (U. of Missouri) Zongming Fei (U. of KY) John Geske (Kettering U.) Deniz Gurkan (U. of Houston) Christos Papadopoulos (Col. State) Violet Syrotiuk (Arizona State U.) Zhi-Li Zhang (U. of MN)
Spring 2013: Jay Aikat (U. of NC) Rudra Dutta (NCSU) Khaled Harfoush (NCSU) Jelena Marasevic (Columbia U) Parmesh Ramanathan (U. Wisc) Violet Syrotiuk (Arizona State U.) KC Wang (Clemson) Michael Zink (U. of MA)
Fall 2012: Rudra Dutta (NCSU) Zongming Fei (U. of KY) Fraida Fund (NY Poly) Kaiqi Xiong (RIT)
Sponsored by the National Science Foundation 35GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Upcoming GENI Events
GENI Engineering Conferences, held three times a yearPlanning & discussion for experimenters, software, infrastructureTutorials and workshops Travel grants to US academics for participant diversity
GENI Summer Camp June ‘14
GEC19
Georgia Tech, Atlanta March 17-19, 2014
Train-the-TAOffered online at the start of each semester
Tutorials at ConferencesIC2E ‘14, etc
Sponsored by the National Science Foundation 36GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Outline
• GENI – Exploring future internets at scale• The GENI Concept• Building GENI• Experimental and Classroom use of GENI• GENI: An experimenter’s view
Sponsored by the National Science Foundation 37GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Slice
– An experiment uses resources in a slice
– Slices isolate experiments
– Experimenters are responsible for their slices
SliceContainer for a collection of resources capable of running experiments
Sponsored by the National Science Foundation 38GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Slice credentials
Clearinghouse and Aggregates
• Clearinghouse: Manages users, projects and slices– Standard credentials shared via custom API or new Common CH API– GENI supported accounts: GENI Portal/CH, PlanetLab CH, ProtoGENI CH
• Aggregate: Provides resources to GENI experimenters– Typically owned and managed by an organization– Speaks the GENI AM API– Examples: PlanetLab, Emulab, GENI Racks on various campuses
Create & Register Slice
Researcher
Aggregate Manager API - listResources - createSliver … Aggregate
ManagerAggregate Resources
users
slices
clearinghouse
projects
Tool
Sponsored by the National Science Foundation 39GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Resource Specifications (Rspecs)
• RSpecs: Lingua franca for describing and requesting resources– “Machine language” for negotiating resources between experiment
and aggregate– Experimenter tools eliminate the need for most experimenters to
write or read RSpec
<?xml version="1.0" encoding="UTF-8"?><rspec xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2 http://www.protogeni.net/resources/rspec/2/request.xsd" type="request" > <node client_id="my-node" exclusive="true"> <sliver_type name="raw-pc" /> </node></rspec> RSpec for requesting a single node
Sponsored by the National Science Foundation 40GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Reserving Resources using RSpecs and the AM API
• Experimenter tools and aggregates talk to each other using resource specifications (RSpecs) and the GENI Aggregate Manager API (GENI AM API)
• Advertisement RSpec: What does an aggregate have?• Request RSpec: What does the experimenter want?• Manifest RSpec: What does the experimenter have?
AggregateManager
Exp
erim
ente
rTo
ol
ListResources(…)
Advertisement RSpec
CreateSliver(Request RSpec, …)
Manifest RSpec
ListResources(SliceName, …)
Manifest RSpec
What do you have?
I have …
I would like …
You have …
What do I have?
You have …
Sponsored by the National Science Foundation 41GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Reserve resources across many aggregates in the same slice
One slice can contain: bare metal machines, virtual machines, VLANs, OpenFlow resources, etc
Backbone #1
Backbone #2
Campus#3
Campus#2
Access#1
CommercialClouds
CorporateGENI suites
Other-NationProjects
ResearchTestbed
Campus My GENI Slice
My slice contains resources from many aggregates.
Sponsored by the National Science Foundation 42GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Putting it all Together: Demo
• Demo– Login to the GENI Portal– Create a slice– Create a sliver at one
aggregate• Two computers (VMs),
connected by a LAN
– Install and run software on the machines
– View output of software– Delete sliver
• Experimenter tool: Flack
server(VMs)
client(VMs)
Sponsored by the National Science Foundation 43GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Start Demo
• Login to GENI Experimenter Portal• Create slice• Launch Flack• Draw topology• Create sliver• Verify sliver creation was successful
Sponsored by the National Science Foundation 44GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
The Demo Experiment in Flack
Sponsored by the National Science Foundation 45GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
The Request RSpec
<rspec type="request” xsi:schemaLocation=“http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/request.xsd” xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.geni.net/resources/rspec/3"> <node client_id="server" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true"> <sliver_type name="raw-pc"> <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/> </sliver_type> <services> <execute command="sudo /local/install-script.sh" shell="sh"/> <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/> </services> <interface client_id="server:if0” /> </node> <node client_id="client" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="false"> <sliver_type name="emulab-openvz"/> <services> <execute command="sudo /local/install-script.sh" shell="sh"/> <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/> </services> <interface client_id="client:if0” /> </node> <link client_id="Lan"> <component_manager name="urn:publicid:IDN+emulab.net+authority+cm"/> <interface_ref client_id="server:if0"/> <interface_ref client_id="client:if0"/> <property source_id="server:if0" dest_id="client:if0"/> <property source_id="client:if0" dest_id="server:if0"/> </link></rspec>
Sponsored by the National Science Foundation 46GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
The Manifest RSpec<rspec type="manifest" …> <node client_id="server" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true" component_id="urn:publicid:IDN+emulab.net+node+pc554" sliver_id="urn:publicid:IDN+emulab.net+sliver+95506"> <sliver_type name="raw-pc"> <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/> </sliver_type> <services> <execute command="sudo /local/install-script.sh" shell="sh"/> <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/> <login authentication="ssh-keys" hostname="pc554.emulab.net" port="22" username="vthomas"/> </services> <interface client_id="server:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc554:eth2" sliver_id="urn:publicid:IDN+emulab.net+sliver+95509" mac_address="0024e87a46fb"> <ip address="10.10.1.1" type="ipv4"/> </interface> </node> <node client_id="client" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="false" component_id="urn:publicid:IDN+emulab.net+node+pc533" sliver_id="urn:publicid:IDN+emulab.net+sliver+95505"> <sliver_type name="emulab-openvz"/> <services> <execute command="sudo /local/install-script.sh" shell="sh"/> <install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/> <login authentication="ssh-keys" hostname="pc533.emulab.net" port="37178" username="vthomas"/> </services> <interface client_id="client:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc533:eth2" sliver_id="urn:publicid:IDN+emulab.net+sliver+95510" mac_address="0262331adfd4"> <ip address="10.10.1.2" type="ipv4"/> </interface> </node> <link client_id="Lan" sliver_id="urn:publicid:IDN+emulab.net+sliver+95508" vlantag="310"> <interface_ref client_id="server:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc554:eth2" sliver_id="urn:publicid:IDN+emulab.net+sliver+95509"/> <interface_ref client_id="client:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc533:eth2" sliver_id="urn:publicid:IDN+emulab.net+sliver+95510"/> <property source_id="server:if0" dest_id="client:if0"/> <property source_id="client:if0" dest_id="server:if0"/> </link></rspec>
Sponsored by the National Science Foundation 48GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
QUESTIONS?
Sponsored by the National Science Foundation 49GENI Introduction – SIGCSE ‘14 – 5 March 2014 www.geni.net
Agenda
Presentation and Demo: An Introduction to GENI (45min)
Hands-on: Getting Started with GENI (45min)
Break (10min)
Hands-on: IPv4 Routing Assignment (50min)
Resources for Instructors (15min)
Miscellaneous GENI Topics & Wrap-Up (15min)