GEC17: Uniform Experimenter Experience

Sponsored by the National Science Foundation

GEC17: Uniform Experimenter Experience

Sunday July 21, 2013 0830-1000Josh Smift, Marshall Brinn

GPO

Sponsored by the National Science Foundation 2

Outline

• Motivation: Why a Uniform Experimenter Experience?

• Recent Progress and Status• Looking Ahead


Motivation

• This topic is intentionally called “Uniform Experimenter Experience”– It doesn’t mean to imply that all aggregates are the

same or all services are or should be the same• But it invites us to focus on the experimenter as

the ultimate consumer of GENI tools, services and capabilities– It is the experimenter’s experience of these services

that we want to be uniform


Motivation [2]

• At the last GEC, we suggested a general rule for approaching the natural differences among GENI services:

• How we do this ‘advertising’ and ‘hiding’ will determine how uniform (and ultimately experimenter-friendly) we will be.

• If a feature is something that is of value to experimenters, advertise it• If not, hide it


Advertising Differences

• Ideally, capabilities should be advertised in advertisement RSpecs and then presented to experimenters by tools.– However, many capabilities cannot be easily rendered

in advertisements or by tools (e.g. containers vs. VM’s)– Differences are often “advertised” in WIKIs which is not

an approach that can be automated or that will scale

If not advertised, value-added differences are likely to be unavailable or invisible to experimenters from most tools.

This leaves us vulnerable to providing a ‘least common denominator’ capability rather than a diversity of valuable features.


Hiding Differences

• We have discussed several approaches towards masking differences between different aggregates and associated resources including– Increasing common RSpec dialects– Tools that speak/translate between dialects– Common post-boot configuration mechanisms– Common images– Limiting the capabilities that can be requested from

tools to those that are common to all


RECENT PROGRESS AND STATUS


Recent progress and status

• Progress on various fronts since GEC 16• Three in particular to talk about here:

– Images:• A more uniform initial environment on nodes• Applicable to custom images or stock ones

– Post-boot scripts:• Good for customization in general• Also good for smoothing over differences

– GENI meta-information:• As input to post-boot scripts• Also possibly useful for experiment configuration

• A generally increasing focus on tools


Images: Overview

• Nice to have the same image on all your nodes

• Four (at least!) kinds of nodes in the racks:– Raw (“bare metal”) nodes (EG and IG)– Xen VMs (IG)– KVM VMs (EG)– OpenVZ containers (IG)

• Slices will often contain two or more of those

• Most PG raw images work on Xen too• Containers: least expensive, but least

flexible


Images: Approaches

• Identical– One set of instructions to build an image– That image then works everywhere– Sounds nice, may not be feasible

• Conversion– Build an image of one type– Follow instructions to convert it to other types– Utah folks have this mostly working for KVM → Xen

• Common source– One set of instructions to build a “GENI image”– Additional instructions turn it into other types– Perhaps appealing if conversion has issues


Post-boot scripts: Overview

• Even multi-rack images may need customization:– OS: Fedora, Ubuntu, etc– AM: ExoGENI/ORCA, InstaGENI/PG/Emulab, etc– Differences in how particular images were created

• Post-boot scripts can help:– Bridge gaps between AMs (expected to narrow)– Smooth over OS differences when necessary– Do other setup stuff that you need

• Run when the node boots– At least once, when the sliver is created– Also every time it reboots, unless you tell it not to


Post-boot scripts: Dingbot

• Dingbot is one example, to bridge current gaps:– Lets you put the same thing in all your rspecs– Handles OS- and AM-specific differences– Extensible to handle other differences– Takes a config file for user-specific stuff

• Thus:

• (http://groups.geni.net/geni/wiki/Dingbot)

<services> <install url="http://www.gpolab.bbn.com/~jbs/dingbot.tar.gz" install_path="/tmp" /> <install url="http://www.gpolab.bbn.com/~jbs/dingbot-jbs.tar.gz" install_path="/tmp" /> <execute shell="/bin/bash" command="sudo /tmp/dingbot/dingbot /tmp/dingbot/dingbot-jbs.json carlin"/></services>

http://groups.geni.net/geni/wiki/Dingbot


Post-boot scripts: Dingbot details

• General:– Checks if it's already run– Logs what it does (separate output and errors)– Sets the hostname (rspec passes a command-line

argument)• OS-specific:

– Installs packages (a list from the config file)– Configures sudo

• AM-specific:– Creates accounts with SSH keys (a list from the config file)– Sets preferred shell on existing accounts

• Easy to add more things


Post-boot scripts: Related work

• Provisioning tools:– Scripts to manage multiple slices at multiple

AMs– Templates for generating AM-specific rspecs

• Orchestration tools:– Zoing: Runs experiments repeatedly out of cron– Configuration management tools like Puppet or

Salt• Graphical I&M-integrated projects:

– GENI Desktop + GEMINI– LabWiki + GIMI


Meta-information: Overview

• Nodes should have access to GENI meta-info– Slice URN, sliver URN, manifest rspec, etc– Definitely useful for post-boot scripts– Potentially also for experiments at run time

• Sometimes this info is dynamic– Sliver may change after creation (UpdateSliver)– Some info not available until creation is complete– So, just stuffing things in files isn't ideal

• Nodes generally can't query the AM directly– You may not want your private key on your nodes– You may not have Omni (esp. if you don't otherwise

use it )


Meta-information: Details

• Proposed new 'geni-info' command on dev list:– Prints output, creates files, or both– Has the same inputs and outputs on all AMs– Source(s) of the data might be AM-specific

• Things to figure out (here? coding sprint?):– What info it should return:

• Manifest rspec for sure – lots of good stuff there• Some things missing, e.g. users and/or keys

– What format for output (JSON? INI? XML?)– Where it should put files (/geni?)

• After that: Write up a detailed spec (GPO)


LOOKING AHEAD


Looking Ahead

• Here are some areas we should consider that would help developers provide a uniform experimenter experience while allowing them to access value-added differentiators– RSpec Factoring. We should revisit the effort to factor

RSpecs into base and extensions so that base document are universally accepted and understood, while extensions allow for capturing resource/aggregate-specific capabilities

– Rspec Services. Solicitation 4 tool builders would benefit from tools/services to help build and parse RSpecs that encapsulate the differences between IG/EG dialects.


Looking Ahead [2]

• We should consider how resource allocation and configuration could be better unified– Common images

• Perhaps a limited set of OS and OS versions?– Common boot scripts– Common run-time environment– Resource-internal services for self-configuration,

discovery

Consider, e.g., GIMI and GEMINI not as a completely separate product built on top of a topology but part of a single process of building the configured topology the experimenter wants.


Looking Ahead [3]

• Uniform Clearinghouse API (under development) should allow for the same tools to speak to different CH’s– Different credentials, different roles/identities, different

policies– Standardize how aggregates interact with multiple

CH’s, federations

• Complete the work of the AM API development– All get to V3 (providing transactional boundaries)– Ratify and move to V4 (implement ‘update’ capability)


DISCUSSION

GEC17: Uniform Experimenter Experience

Documents

Transcript of GEC17: Uniform Experimenter Experience