1 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok A PlanetLab (Europe) Short Tutorial Serge...

1 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok

A PlanetLab (Europe) Short Tutorial

Serge Fdida

Université Pierre & Marie Curie, LIP6Paris, France

An Open Federated Laboratory Supporting Network Research for the Future Internet

PlanetLab (-Europe)

An open, shared platform for developing, deploying, and

accessing planetary scale applications

PlanetLab short Tutorial

● Introduction Starting from the Customer end PlanetLab - what is it? First steps PlenatLab architecture

● Operation Operation of the testbed? Monitoring

● Beyonf PlanetLab Federation

In this tutorialIntroduction to PlanetLab

● Tutorial Step 1: Register with the system

● Tutorial Step 2: Generate an SSH key

PlanetLab Architecture

● Tutorial Step 3: Adding a Node

● Tutorial Step 4: Creating a Slices

● Tutorial Step 5: Assign users to a slice

● Tutorial Step 6: Adding nodes to a slice

PlanetLab Tools

● Tutorial Step 7: Log in to a sliver

● Other things to do with on a sliver

Overview

● Global distributed system infrastructure platform for long running services testbed for network experiments

● Launched in March 2002

Why Use PL?

● Example: Content delivery network.

● Deploys content and allows access

● How do I build a content deliver network?

● Proof: Does it work?

● Start with a few nodes, for content delivery system. Build up. Test to see if the system works as the system builds and grows, and is a large scale testbed.

About PlanetLab

• +1011 nodes around the world

– 41 countries

– 487sites (universities, research labs)

– 5030 registered users, 630 active slices

– more than 1000 researchers

• A collection of machines distributed over the globe

– Most of the machines are hosted by research institutions

– All of the machines are connected to the Internet Software

– All PlanetLab machines are administered by a system called MyPLC

• The software is based on Fedora Core 8

• Content Distribution Network

~330 (open) caching proxy servers

Open to all users (see URL)

• Highly available (after lots of work!)

• Spawned many subprojects / services:

– CoBlitz, scalable distribution of large files.

– CoDeploy, efficient synchronization for slices.

– CoDNS, fast and reliable name lookup.

– CoMon, node monitoring for PlanetLab

– CoTest, login debugging tool for nodes

– PlanetSeer, distributed network anomaly tracing

• Illustrates how deployment of a real service spurs research

– http://codeen.cs.princeton.edu/

Example: CoDeeN (Princeton)

PlanetLab is not…

● A distributed supercomputer

● A simulation platform

● An Internet emulator

● An arena for repeatable experiments

● Completely representative of the current Internet

● Grid

Organizational Challenges

● PlanetLab Consortium Licensing Agreement

● Organization is responsible for the activities of their researchers

● Users may not use PlanetLab to disseminate copyright material

● PlanetLab tracks resources usage and maintains a list of published intentions

● Otherwise, PlanetLab is a free for all

The PlanetLab Europe Consortium

● Membership

● The Consortium includes eight membership levels

• Charter (300k € annual dues)

• Full (75k € annual dues)

• Associate (25k € annual dues)

• Sponsor (10k € annual dues)

• SME (1k € annual dues)

• Director (no annual dues)

• Academic (no annual dues)

• FP7 (no annual dues)

Using PlanetLab

● Central Website that manages All accounts All nodes All resources

● Registering with PLC (your PlanetLab Central)

● 3 PLC PL USA (planet-lab.org) PL Europe (planet-lab.eu) PL Japan (planet-lab.jp)

Architecture (people)

● 3 Roles

PI (Principal investigator)■ Manage users / slices of his site

Tech■ Manage nodes

User■ Use slices and nodes to launch experiments

● Special role for administration / Management

PI’s Roles and Responsibilities• Oversight. PIs are responsible for overseeing all slices that

they create on behalf of the users at their site.

• Account management. PIs can:

– Enable, disable, and delete user accounts.

– Create slices.

– Delete slices.

– Assign users to slices.

– Allocate resources to slices.

• Node management. PIs are responsible for the

physical maintenance of the nodes at their site

PlanetLab Architecture

Terminology

● Site A: site is a physical location where PlanetLab nodes are located (e.g. Fraunhofer Institute or UCL).

● Node A: node is a dedicated server that runs components of PlanetLab services.

● Slice A: slice is a set of allocated resources distributed across PlanetLab. Slices are assigned to some number of PlanetLab nodes. Slices have a finite lifetime and must be periodically renewed to remain valid.

● Sliver A: sliver is a slice running on a specific node. You can use ssh to login to a sliver on a specific node.

Architecture (Hardware)

UPMCINRIA

Private MyPLC

Princeton (PLC)SVN

Backup MyPLCTutorial

Private MyPLC

Backup

MyPLCMail

Mailing list

Monitoring

Backup

Architecture: general view

DB server(postgresql)

Boot server(apache)

Web server(apache)

API server(apache,

Mod_python)

HTTP(S)

XML-RPCHTTPS

XML-RPC

XML-RPCHTTPS

XML-RPC

Virtualization

● Why virtualization

Security■ A hacked account is « locked » into a virtual machine

Isolation■ Users cannot see each other

■ Fair share of ressources

Give root access / own environment■ yum install, ...

Ease of install / shutdown

Virtualization

Context0 (root context)

site_adminsite_adminsite_adminSlice1

site_adminSlice3 site_adminSlice2

For PL's adminsFull access

For local techRestricted access

For users

Slices – hujiple_isis

Slices – upmcple_paristr

Slices

Security (user)

Private SSH key

PLE (MyPLC)

Public SSH key

Trust Relationships

• 1) PLC expresses trust in a user by issuing it credentials to access a slice

• 2) Users trust to the PLC to create slices on their behalf and inspect credentials

• 3) Owner trusts PLC to set users and map network activity to right user

• 4) PLC trusts owner to keep nodes physically secure

ServiceDeveloper

(User)TPLC

NodeOwner

Security Architecture

● Node Operating System

– isolates slivers

– audits behavior

● PlanetLab Central (PLC)

– remotely manages nodes

– bootstrap services to instantiate and control slices

– monitor sliver/node health

Node Architecture

Virtual Machine Monitor (VMM)

Hardware

LocalAdminSlice

VM1 VM2 VMn&NodeMgr

VMM• Linux

– significant mindshare

• Vserver

– scales to hundreds of VMs per node (12MB each)

• Scheduling

– CPU

fair share per sliver (guarantees possible)

– link bandwidth

fair share per sliver

average rate limit: 1.5Mbps (24 hour bucket size)

peak rate limit: set by each site (100Mbps default)

– disk

5GB quota per sliver (limit runaway log files)

– memory

no limit

pl_mom resets biggest user at 90% utilization

Auditing & Monitoring• PlanetFlow

– logs every outbound IP flow on every node

retrieves packet headers, timestamps, context

ids (batched)

– used to audit traffic

– aggregated and archived at PLC• SliceStat

– has access to kernellevel/ systemwide information

– used by global monitoring services

– used to performance debug services

PlanetLab Tutorial

A step by step guide

First steps to using PlanetLab

1. Register with the system

2. Create an ssh key

3. Create a slice

4. Add nodes to the slice

5. Describe the slice

6. Log into a sliver

Step 1: Register with the system

● Go to https://www.planet-lab.eu

● Click on Create an account

● Fill in personal information first name, last name, title telephone number

● Choose a password and valid email as login

● Select your site Université Pierre et Marie Curie

● Click Register

User Registration

User registers on website

User receives email to verify registration

User verifies email address

PI receives request to enable

PI enables account

Approve?

User receives confirmation email

Rejected

PI Enables Account

● Log in as user

● Choose Enable under CHOOSE ACTION menu

PI Enables Account

Account approval (as PI)

Principal Investigator Account

Account approval (as PI)● Choose Enable User name

VIDEO – Register with PLE

Step 2: Generate an ssh key

● Access to resources is secured through public key encryption

● You must upload your public ssh key to use PlanetLab

Step 2: Generate an ssh key

● PlanetLab uses 1024bit RSA keys for authentication on nodes

● Generate an ssh key using OpenSSH:

sshkeygen –t rsa f /.ssh/id_planetlab

● Please use a secure passphrase

VIDEO –

● Generating and uploading an ssh key

Step 3: Adding nodes● Only a Tech can add a node to a site

● Nodes MUST Have a visible, public DNS name

Be outside of a NAT or FIREWALL

Recommended to not have an IDS or Proxy upstream of the node

Recommended Configurations● Recommended Configuration

More than 1 GB RAM

At least 100 GB hard disk

At least 1 Mb/sec connection to the Internet

P4 or better CPU

External PCU optional

How to add a node

● Tech registers the node with the PLC

● Download iso from the PLC

● Burn to CD or copy to USB

● Insert media on node

● Boot from media

● Answer Yes to installation request

● Wait 60 minutes or more while the node is formatted and installed (depending on network connection)

Adding nodes

● Fill in form, Click Add

VIDEO – Adding a node

Step 4: Creating a slice

● Actually, ask your PI to create a slice…

● Enter slice name <site>_<slicename>

● Enter project url

● Enter description

● Assign users to slice

● Click Add Slice

Your Slice is ready to serve

VIDEO – Cretaing a slice

By now…

● You have a user account

● Your PI has approved it

● Your PI has created a slice

● Your PI has assigned you as a user of the slice

● Next step: add nodes to the slice

Adding nodes to your slice

● Page for adding/removing nodes

– Select site to see nodes to add

– Shown are all nodes currently in slice

– Nodes can be removed

● For your first time, it is recommended:

– Add a couple of nodes, one nearby

● Note: Changes will take ~5 minutes to propagate on the tutorial system. It may take up to two hours on production PlanetLab system.

Step 6: Adding Nodes to a slice

Step 6: Adding nodes to a slice

● Search and select nodes to add easly

● Nodes can be removed

VIDEO – Adding nodes to a slice

Step 7: Log in to a sliver

● Your user name is your slice name

● E.g., with OpenSSH:

Populating your sliver

sudo yum install gcc

● sudo allows you to be root in your sliver

● yum is the redhat package manager

● You can use any RPM packages you want in your sliver

● Caveat Emptor: yum may take up to one hour to initialize and begin installation of your packages

Deploying to a sliver• scp -l huji_dimes \

-i ~/.ssh/id_rsa -r \

localdir evgsics7.sics.se:

• Copy localdir to the remote machine

• rsync -a -e “ssh -l huji_dimes” \

localdir evgsics7.sics.se:

• Update the remote directory. Copies only those files that are different

Deploying Software to Your Nodes

● Copy binaries to the node to run

– scp or rsync works for small numbers of nodes

– Ensure library dependencies are satisfied

References● PlanetLab official Web

site:http://www.planetlab.org/

● L. Peterson, S. Muir, Timothy Roscoe, and Aaron

● Klingaman PlanetLab Architecture: An Overview. Technical Report, PlanetLab, May 2006

● L. Peterson and T. Roscoe. The Design Principles of PlanetLab.

● Operating Systems Review (OSR), 40(1):11.16, Jan. 2006.

Backup

Operations

PlanetLab (PLC) & PlanetLab Europe (PLE)

An Open Federated Laboratory Supporting Network Research for the Future Internet

Using PlanetLab

● Central Website that manages All accounts All nodes All resources

● Registering with PLx (your PlanetLab Central)

● 3 PLC PL USA (planet-lab.org) = PLC PL Europe (planet-lab.eu) = PLE PL Japan (planet-lab.jp) = PLJ

PlanetLab Europe overview

Main operations centre EverLab opsPLC ops

Using PlanetLab Europe

NodesResearcher

Slicerequest

Requestpropagation

Researcher

Sliceallocation

Researcher

Sliceallocation

Researcher

Sliceallocated

Researcher

Slicelogin

Researcher

Runexperiment

Researcher

Externalusers

Researcher

PlanetLab Europe overview

What we manage

Growth of PlanetLab Europe

PLC sitemanaged byPLE

Addition ofwireless(new)

Addition ofemulation(new)

Monitoring

● Who needs monitoring?

Operations

Scientific research purposes

European community

● Operations monitoring:

Health / availability

New features' deployments

Load of the system

Tools:■ PlanetFlow

■ Monitor

■ EverStat

■ PLE Graphs

Auditing & Monitoring● PlanetFlow

– logs every outbound IP flow on every node

retrieves packet headers, timestamps, context

ids (batched)

– used to audit traffic

– aggregated and archived at PLC● SliceStat

– has access to kernellevel/ systemwide information

– used by global monitoring services

– used to performance debug services

Monitoring examples

Federation

What is federation?

Site 1

Site 2

Ressources Ressources

What is federation?

Site 1

Site 2

Ressources Ressources

Classical PLE

● Registry

Users, slices, components, authorities

Exports registry interface

● Slice Manager

Create & control slices

Export research interface

● Aggregate Manager

Responsible for a set of components

Exports slices & management interface

● Component Manager

Controls a component

Exports slices & management interface

NodeCM

Classical PLE

CM CM CM

● Slice interface Create & control slices/silvers

● Registry interface Bind & resolve naming info

● Managment interface Query & reboot components

● Uber research interface Slice interface, ...

Classical PLE

CM CM CM

Alternative SM

CM CM CM

EMULAB

CM CM CM

EMULAB

Emulab (users and slices in PLC)

CM CM CM

EMULAB

CM CM CM

Vini PLC (single SM)

CM CM CM

VINIUSER

CM CM CM

Full federation

CM CM CM

Concluding remarks● Building a facility is a major challenge

Complex process, High risk, non technical issues (IPR, Legal,)

● FIRE / Onelab is about: Supporting two complementary dimensions (NS & Exp) Enabling different federations – not one size fits all Basing on an existing ecosystem with an international community

● Onelab is already: Up and running! Independent and Federated Highly visible worldwide, seen as a peer wrt other testbeds Cooperation with « Pilot » projects (PSIRP, ANA, Haggle, 4Ward,

FEDERICA) – looking for new partnerships. Aggregate tools of disperse communities

Coffee Break

Tutorial Site

The tutorial (pdf slides) are available at:

http://www.planet-lab.eu/tutorials

The three tutorials are:

PLE Basics, https://www.planet-lab.eu/files/PlanetLab_Basics.pdf

Technical Overview, https://www.planet-lab.eu/files/PlanetLab__Tech_Overview.pdf

Adding a Node, https://www.planet-lab.eu/files/PlanetLab_Adding_a_Node.pdf

The video tutorials are available at:

http://www.planet-lab.eu/doc/tutorials/videos/

Questions

1 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok A PlanetLab (Europe) Short Tutorial Serge...

Documents

Transcript of 1 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok A PlanetLab (Europe) Short Tutorial Serge...

Future Internet Research in Europe, FIRE, and the OneLab ... GEC3...5 Framework Programme 1998-2002 2002-2006 Creating a user-friendly information society (IST) 3.6 G€ / 4 years

Orientation Session of AsiaFI School Session Chairs: Kideok Cho (Seoul National Univ.) Wang Yang (Tsinghua Univ.)

PlanetLab and OneLab Presentation at the GRID 5000 School 9 March 2006

EXPLORING THE SIMULATION OF DYNAMIC PARTIAL ...onelab-eg.com/wp-content/uploads/2019/02/PDF-1.pdfDynamic Partial Reconfiguration, Network-on-Chip, Fields Programmable Gate Array Summary:

2010 ASIAFI: NDN applicability to automotive scenarios.asiafi.net/meeting/2010/summerschool/p/wakikawa.pdfCooperative Intersection Collision Avoidance System ... transit vehicle tracking,

OneLab An Open Federated Laboratory to evaluate the possible futures of the Internet

New CAD features, Gmsh 3.0 and API - ONELAB · Set OpenCASCADE internal CAD modeler Explicit tags Delete object and tool. Third Gmsh Workshop - Lanzarote March 29-31, 2017 14 New

Planetlab Tutorial - unina.it · Planetlab Tutorial Alessio Botta 1 “Onelab - Planetlab Europe” information day, Napoli, ... Università di Napoli “Federico II” ... Introduction

Incubadora Digital Faithful - OneLab

Asia Future Internet Workshop OneLab Can we build a test-bed to explore the “future” Internet?

Future Hybrid Network in Korea and Beyond 2008. August 25th Dongkyun Kim, KISTI mirr@kisti.re.kr AsiaFI Testbed Workshop 2008 in Jeju.

Southeast University Dept. of Cs. & Eng. 2008.8 AsiaFI School Wang Yang Southeast University August 2008 FAME : Factor Analysis Based Metrics Exploring.

1 - OneLab - CPqD– September 23, 2009 – Campinas Brazil OneLab International Workshop: New Architectures for Future Internet OneLab Can we build a test-bed.

Serge Fdida – CNRS – Sept’05 French (Network) Security Research Activities Serge Fdida University Paris 6 & CNRS Contributions from Michel Riguidel (ENST)

Vee Onelab Users Guide

OneLab: Federating Testbeds

Development of PMIPv6 based 6LoWPAN Sensor Node Mobility Scheme Jin Ho Kim, Rim Haw, Choong Seon Hong Kyung Hee University, KOREA 2010.02.25 AsiaFI School.

M-HBH Efficient Mobility Management in Multicast Rolland Vida, Luis Costa, Serge Fdida Laboratoire d’Informatique de Paris 6 – LIP6 Université Pierre et.

Test-Bed for testing scenarios of new DTN architecture AsiaFI 2008 August, 2008 Multimedia and Mobile Communication Lab.

Report on Onelab/2 Activities Future Internet Research and Experimentation Report on Onelab/2 Activities Serge Fdida sf/ Université.