Sponsored by the National Science Foundation Systematic Experimentation in GENI Sarah Edwards GENI...
-
Upload
ronald-french -
Category
Documents
-
view
218 -
download
0
Transcript of Sponsored by the National Science Foundation Systematic Experimentation in GENI Sarah Edwards GENI...
Sponsored by the National Science Foundation
Systematic Experimentation in GENI
Sarah Edwards
GENI Project Office
Sponsored by the National Science Foundation 2Systematic Experimentation – GEC 20 – June 23, 2014
General Principle #1: One thing at a time
Experimentation 101 and Debugging 101
Only change one thing at a time
Examples:• Software/image• Configuration• Number of nodes/links• Geographic distribution of nodes/links
Sponsored by the National Science Foundation 3Systematic Experimentation – GEC 20 – June 23, 2014
General Principle #2: Start Small
Start by building smallest possible topology by hand.
Automate as needed.
Test and measure as you go.
client server
host
OVSswitch
host
host
router
router router
node
Sponsored by the National Science Foundation 4Systematic Experimentation – GEC 20 – June 23, 2014
General Principle #3: Save what you do
• Log all of your experimental artifacts for every experiment that works– RSpec – image– install script– custom software– measurements– etc
• Use version control to store your artifacts• Always know the last configuration that worked
Sponsored by the National Science Foundation 5Systematic Experimentation – GEC 20 – June 23, 2014
Process Recommendation
A. Build (smallest possible) topology by hand at a single aggregate
B. Automate topology creation at a single aggregate
C. Orchestrate and Instrument
D. Increase scale
E. More nodes
F. More aggregates
Automate
Sponsored by the National Science Foundation 6Systematic Experimentation – GEC 20 – June 23, 2014
Design Choices
• Custom images vs install scripts vs both• Single aggregate vs multi-aggregate• Which aggregates?• Bare metal vs virtual machine• What tools are you comfortable with?
There are no right answers. Pick what’s needed for your experiment.
Sponsored by the National Science Foundation 8Systematic Experimentation – GEC 20 – June 23, 2014
“How To” pages
• Listed under the “Experimenters” section
• Each “How To” is a short descriptions of how to do various tasks
• New entries being added all the time
Sponsored by the National Science Foundation 9Systematic Experimentation – GEC 20 – June 23, 2014
Finding other resources
• GENI wiki– Pages for Instructors and Experimenters
http://groups.geni.net/geni/wiki
Sponsored by the National Science Foundation 11Systematic Experimentation – GEC 20 – June 23, 2014
Systematic Experimentation
Running an experiment on a testbed requires best
practices and methodology from a combination of:– Science (Scientific method)
– Programming
– System Administration
Today, talking about strategies and techniques for
bringing up experiments in a systematic way.
Sponsored by the National Science Foundation 12Systematic Experimentation – GEC 20 – June 23, 2014
Why experiment systematically?
Debugging Easier to debug experiments. Easier to get help debugging. Especially for complex experiments.
Scalability and Repeatability A well designed experiment will be repeatable and systematically scalable
Valid Develop a solid understanding of and validate your experimental setup
Rigorous Perform a scientifically rigorous (and publishable) study
Sponsored by the National Science Foundation 13Systematic Experimentation – GEC 20 – June 23, 2014
GENI-FIRE 150 node topology
Courtesy of Thierry Rakotoarivelo, NICTA, GEC 20 Demo
Clear plan AutomationScale and
Visualization
Sponsored by the National Science Foundation 14Systematic Experimentation – GEC 20 – June 23, 2014
Virtual Computer Networking Lab
Courtesy of Cong Wang and Divya Bhat, UMass Amherst, GEC 20 Demo
Start small
Sponsored by the National Science Foundation 15Systematic Experimentation – GEC 20 – June 23, 2014
Intelligent Data Movement System
Courtesy of Ezra Kissel, Indiana University, GEC 20 Demo
Long-lived slicefor stitched, shared VLAN
Dynamically add/remove nodes
as needed
Sponsored by the National Science Foundation 16Systematic Experimentation – GEC 20 – June 23, 2014
Why doesn’t this work?(Anonymous) Past Bug Reports
repeatable
incremental
test controller
in one AM?
GRE works
Stitch
test controllerin one AM
Multi-site Hadoop
Used control plane to send high volume data between
racks
Interfere with rack !
OpenFlow multi-site
OpenFlow multi-site
Sponsored by the National Science Foundation 17Systematic Experimentation – GEC 20 – June 23, 2014
Process: Start Small
A. Build smallest possible topology by hand at a single aggregate
B. Automate topology creation at a single aggregate – Store RSpecs, software, etc in version control – TOOLS Use install scripts and custom images– Determine how to update software on your nodes
Repeat A/B as needed.
C. Orchestrate and Instrument your experiment procedureTOOLS GENI Desktop and LabWiki
Automate as needed.
worker
master
worker
node
Sponsored by the National Science Foundation 18Systematic Experimentation – GEC 20 – June 23, 2014
Process: Scale
D. Increase scale (number of nodes, volume of traffic, etc) TOOLS Flack for small scale. geni-lib for larger scale
client
server
client
client clientclient
client server
Sponsored by the National Science Foundation 19Systematic Experimentation – GEC 20 – June 23, 2014
Process: Split across aggregates
E. Split an existing topology across second aggregate – How connect across aggregates matters (stitching vs OF etc)
TOOLS stitcher for VLAN stitching and GRE tunnels. ExoSM for Intra-ExoGENI stitching. Flack for GRE tunnels.
F. Split across multiple aggregates
client client
server
client clientclient
Colors represent different aggregates
Inter-aggregate link