Building Clouds with OpenNebula2.2

Ruben S.Montero

dsa-research.org | OpenNebula.org

Distributed Systems Architecture Research Group

Universidad Complutense de Madrid

Centro de Supercomputación de Galicia

May 2011, Spain

Copyright 2002-2011 © OpenNebula Project Leads (OpenNebula.org). All Rights Reserved.Creative Commons Attribution Share Alike (CC-BY-SA)

Building Clouds with OpenNebula 2.2

OpenNebula.org Creative Commons Attribution Share Alike (CC-BY-SA)

Day one

Introduction to Cloud Computing

Installing OpenNebula 2.2

Configure your Cloud (storage, hypervisor and network)

Administration of an OpenNebula Cloud (hosts, users)

Image Management

Networking

Basic VM Managment

Day two

Sunstone GUI

More VM Managment

Configuring an Hybrid Cloud with Amazon EC2

Public Cloud interfaces: The EC2 Query API

Advance Topics: Adapt OpenNebula 2.2 to your datacenter

Contents

Schedule

Private Cloud

Hybrid Cloud

Public Cloud


Describe the benefits and characteristics of virtual

infrastructures and IaaS clouds

Describe the characteristics and architecture of the different

clouds that can be deployed with OpenNebula 2.2

Plan and architect a private cloud

Design, Use and Manage Cloud Applications

Build public and hybrid clouds

Adapt OpenNebula 2.2 to your datacenter

Course Overview

What will you learn?

This box contains interesting messages

This is a console output, for hands on, checking configuration files

# This is the root prompt

$ This is oneadmin prompt

Ruben S.Monterodsa-research.org | OpenNebula.org

Distributed Systems Architecture Research GroupUniversidad Complutensede Madrid



“An Introduction to Cloud Computing”

• Cloud Computing Disciplines

• Infrastructure as a Service

• Chanllenges of IaaS Clouds

• History of OpenNebula.org

• Technical Overview of OpenNebula


What Who

On-demand access to any application

End-user(does not care about hw or sw)

Platform for building and delivering web applications

Developer(no managing of the underlying hw & swlayers)

Delivery of a rawcomputer infrastructure

System Administrator(complete management of the computer infrastructure)

Cloud Computing Disciplines

An Introduction to Cloud Computing

Platform as a Service

Infrastructure as a

Service

Physical Infrastructure

Software as a Service


Infrastructure as a Service (IaaS)


• Simple Web Interface

• Raw Infrastructure Resources

• Pay-as-you-go (On-demand access)

• Elastic & “infinite” Capacity

Public Cloud








Public Cloud

A “Public Cloud behind the firewall”

• Simplify internal operations

• Dynamic allocation of resources

• Higher utilization & operational savings

• Security concerns

Private Cloud








Public Cloud






Private Cloud

• Suplement the capacity of the Private Cloud

• Utility Computing dream made a reality!

Hybrid Cloud









Private Cloud





Public Cloud

• Suplement the capacity of the Private Cloud

• Utility Computing dream made a reality!

Hybrid Cloud


Challenges of an IaaS Cloud


I’m using virtualization/cloud, and plan a private Cloud (BUT’s)

Where do/did I put my web server VM?

Monitoring & Scheduling

Can I use hypervisor X?

Virtualization

ºUniform management layer that orchestrates multiple technologies

How do I provision a new VM?

Image Management & Context

How do I create a new disk?

Storage

Who have access to cloud (and What)?

User & Role Management

How do I set up networking for a multitier service?

Network & VLANsHow can I manage the distributed infrastructure?

Interfaces & APIs


History of OpenNebula.org


2005 2008 2009 2010 2011 2012

dsa group doing

research…

TP v1.0 v1.2 v1.4 v2.0 v2.2 v2.4

4000

downloads/month

European Funding

• Develop & innovate

• Support the community

• Collaborate

Third party scalability

tests: 16000 VMs

Commercial Support


History of OpenNebula.org: Sample Users


Organizations Building Clouds for Development, Testing and Production

16,000 VMs!

Projects Building an Open Cloud Ecosystem Around OpenNebula


Technical Overview of OpenNebula: Vision & Design Philosophy


• One solution can not fit all data-center, requirements and constraints

• Open, felxible and extensible architecture that allows multiple

components to be orchestrated

• Ready for production

• Massively scalable deployments

• Open Source – Apache License v2.0

• Provide basic components, but allow them to be easily replaceable


Technical Overview of OpenNebula: Key Components


VM Networking

• VLANs

• Firewall hooks

• User defined

Hosts: Cluster workernoes to run VMs.

• Multiple hypervisors defined per host

• Grouped in logical clusters

• Custom monitoring probes and technologies

Storage

• NAS

• SAN

• Custom (bittorrent, ssh…)

Interfaces & API

• CLI (local/remote)

• API (java, ruby bindings)

• Sunstone

• Cloud (EC2, OCCI)

User & Roles

Image Repository of VM disks

• ACLs (public + private)

• OS and Data types (persistent)

• Multiple storage backends





“An Introduction to Cloud Computing”

• Cloud Computing Disciplines

• Infrastructure as a Service

• Chanllenges of IaaS Clouds

• History of OpenNebula.org

• Technical Overview of OpenNebula





Preparing the Cloud for OpenNebula

• Overview of OpenNebula Components

• Runtime & Compilation Requirements

• Users & File-System

• Storage for the Private Cloud

• Networking for the Private Cloud

• Hypervisor Configuration

• Checklist


Component Overview

Preparing the cloud for OpenNebula Executes the OpenNebula Services

Usually acts as a classical cluster front-end

Provides physical resources to VMs

Must have a hypervisor installed

Modular components to interact

with the cluster services

Types: storage, monitoring,

virtualization and network

Repository of VM images

Multiple backends (LVM, iSCSI..)

Usually in a separate host

The same host can be can be a

the front-end and a node


Runtime Requirements

Cluster Front-end Choose your installation mode

system wide (/usr, /etc...)

self-contained (under $ONE_LOCATION)

Install software dependencies (runtime)

Runtime (also ssh and openssl)# apt-get install ruby

# apt-get install sqlite3

# apt-get install libxmlrpc-c3

Check for additional notes at:

http://www.opennebula.org/documentation:rel2.2:notes


Building Requirements

Cluster Front-end

# apt-get install libxmlrpc-c3-dev libsqlite3-dev libssl-dev

# apt-get install build-essential g++ scons

# apt-get install ruby-dev rubygems rake

The following will give you a faster XML parser (faster CLI)

# apt-get install libexpat1-dev libxml-parser-ruby1.8

# apt-get install libxslt1-dev libxml2-dev

# gem install xmlparser

# gem install nokogiri

SET SYSTEM LOCALES TO ENGLISH (oneadmin should be enough...)

# cat /etc/default/locale

LANG="en_US.UTF-8“

Development libraries and tools (only needed to build OpenNebula)


Users & File-System Layout

Cluster Front-end

The Users of the private cloud:

oneadmin: Account to run the daemons, manage the system and do all the low-level operations (e.g. start VMs, move images...).

Users: create and manage their own VMs and networks. Need to be

defined in OpenNebula

Installation layout

We will use the /srv/cloud directory to place the OpenNebula software

/srv/cloud/one will hold the OpenNebula installation

/srv/cloud/images will do our “image repository” in the course



Cluster Front-end

Installation layout

# tree /srv

/srv/

`-- cloud

|-- images

`-- one

|-- SRC

The oneadmin account must be created system wide (i.e. front-end

and all the nodes) you can use NIS, or a local account with the same

ID's in all the hosts. Users do not need a UNIX account in the nodes,

nor in the front-end.



Cluster Front-end

# groupadd -g 3000 cloud

# mkdir /srv/cloud

# chgrp cloud /srv/cloud

# chmod g+ws /srv/cloud

# mkdir /srv/cloud/images

# chmod g+w /srv/cloud/images

# useradd -d /srv/cloud/one -g cloud -u 3000 -s /bin/bash -m oneadmin

Create the file-system hierarchy with the oneadmin account

$ id

uid=3000(oneadmin) gid=3000(cloud) grupos=3000(cloud)

We will place the OpenNebula source code in SRC

$ mkdir SRC

$ cd SRC

$ wget http://dev.opennebula.org/attachments/download/339/opennebula-

2.2.tar.gz

Hands on: Create the installation dirs and oneadmin account


Image Repository: Any storage medium for the VM images

(usually a high performing SAN). In this course a fs-based repo.

Cluster Storage

OpenNebula supports multiple back-ends (e.g. LVM for fast cloning)

VM Directory: The home of the VM in the cluster node

Stores checkpoints, description files and VM disks

Actual operations over the VM directory depends on the storage medium

Should be shared for live-migrations

You can go on without a shared FS and use the SSH back-end

Defaults to $ONE_LOCATION/var/$VM_ID

Storage for the Priv ate Cloud

Component Overview



Dimensioning the Storage... Example: A 64 core cluster will typically run around 80VMs,

each VM will require an average of 10GB of disk space. So you will need ~800GB for

/srv/cloud/one, you will also want to store 10-15 master images so ~200GB for

/srv/cloud/images. A 1TB /srv/cloud will be enough for this example setup.

In this course we will

use NFS to share the

VM directories

The Image Repository

is /srv/cloud/images

Example, a shared FS architecture



Configuring NFS backend

# apt-get install nfs-kernel-server

Export /srv/cloud to your nodes

- only need /srv/cloud/one/var

- we also export $HOME of oneadmin for easy SSH key configuration

- No need to export /srv/cloud/images

# vim /etc/exports

/srv/cloud 193.144.33.YY(rw,async,no_subtree_check,no_root_squash)

# service nfs-kernel-server restart

# service ufw stop

# iptables -F

Hands on: set up the storage


OpenNebula management operationsuses a ssh connections, it does notrequire a performing NIC

Image traffic, may require themovement of heavy files (VM images,checkpoints). Dedicated storage linksmay be a good idea

VM demands, consider the typical requirements of your VMs. Several NICs to support the VM traffic may be a good idea

OpenNebula relies on bridgenetworking for the VMs

Networking for the Private Cloud

Component Overview



Cluster Worker-nodes

Install software dependencies

We need SSH daemon running in the cluster nodes (check it!)

Runtime dependencies:

Users

Create the oneadmin account (use same UID and GID)

Set language environment to english

# apt-get install ruby

# groupadd -g 3000 cloud

# useradd -d /srv/cloud/one -g cloud -u 3000 -s /bin/bash oneadmin

# cat /etc/default/locale

LANG="en_US.UTF-8“


Storage for the Private Cloud

Example, Configuring NFS backend in the Worker-nodes

Storage: recreate the installation layout and configure NFS to mount

the VM dirs

# mkdir /srv/cloud

# chmod g+ws /srv/cloud

# chgrp cloud /srv/cloud

# ls -l /srv/

total 8

drwxrwsr-x 2 root cloud 4096 2011-05-02 14:09 cloud

# apt-get install nfs-common

# vi /etc/fstab

193.144.33.x:/srv/cloud /srv/cloud nfs

soft,intr,rsize=32768,wsize=32768,rw 0 0

Note: Add the previous in just one line

# service stop ufw (beware of any firewall rule)

# iptables –F

# mount –t nfs -a



Configuring SSH access

• Enable password-less SSH access to cluster (oneadmin)

Do not protect the private key with a password

$ ssh-keygen

Generating public/private rsa key pair.

...

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

Tell ssh client not to ask to add hosts to known_hosts (optional)

$ cat /srv/cloud/one/.ssh/config

Host *

StrictHostKeyChecking no

$ ssh 193.144.33.yy

You may need to exchange keys with the nodes (not here as we share /srv/cloud/one the

oneadmin home, and so the ~/.ssh directory )


Hypervisor Configuration

Example, configuring KVM in the Worker-nodes

Installing the Hypervisor

OpenNebula supports KVM, Xen and Vmware (even simultaneously).

This course applies to KVM and Xen

Refer to the hypervisor documentation for additional (and better

information) on setting up them.

Setting up KVM and libvirt (Ubuntu 10.04)

Install the packages (should be already installed)

#apt-get install qemu-common qemu-kvm libvirt-bin


Add oneadmin to the libvirt group

Test the installation for the oneadmin account

Fix apparmor issue with libvirt

Hypervisor Configuration

Example, configuring KVM in the Worker-nodes

# usermod -G kvm,libvirtd oneadmin

$ virsh –c qemu:///system list

Id Name State

----------------------------------

# tail /etc/apparmor.d/abstractions/libvirt-qemu

# https://launchpad.net/bugs/457716

#include <abstractions/private-files-strict>

owner @{HOME}/ r,

owner @{HOME}/** rw,

/srv/cloud/one/var/** rw,

# service apparmor restart



Network Configuration in the Worker-nodes

Setting up KVM and libvirt (Ubuntu 10.04)

Networking for this course

Disable virbr0

#rm /etc/libvirt/qemu/networks/autostart/default.xml

#ifconfig virbr0 down

#brctl delbr virbr0

#service libvirt-bin restart

br0

VM VM

192.168.0.1 192.168.2.1

193.144.33.x



Network Configuration in the Worker-nodes

Disable ubuntu network manager

Check the network configuration (ifconfig, brctl show)

# vim /etc/network/interfaces

auto lo

iface lo inet loopback

auto br0

iface br0 inet static

address 193.144.33.150

netmask 255.255.255.192

network 193.144.33.128

broadcast 193.144.33.191

gateway 193.144.33.129

bridge_ports eth0

bridge_stp on

bridge_maxwait 0

bridge_fd 0


Installation Checklist

Preparing the cloud for OpenNebula





Preparing the Cloud for OpenNebula

• Overview of OpenNebula Components

• Runtime & Compilation Requirements

• Users & File-System

• Storage for the Private Cloud

• Networking for the Private Cloud

• Hypervisor Configuration

• Checklist





Installing & Configuring OpenNebula

• Installing OpenNebula 2.2

• Configuring OpenNebula 2.2

• Managing Hosts & Clusters

• Managing Users

• Logging and Debugging Information



Compiling the Software Grab the source code and compile it! (oneadmin)

Install the software in /srv/cloud/one (ONE_LOCATION)

Check and explore the installation tree

~/SRC$ wget http://dev.opennebula.org/attachments/download/339...

~/SRC$ tar xzvf one-2.2.tar.gz

~/SRC$ cd opennebula-2.2/

~/SRC$ scons

$ export ONE_LOCATION=/srv/cloud/one/

$ ./install.sh -d $ONE_LOCATION

Check install.sh -h for other options

~$ ls -F

bin/ etc/ examples.desktop include/ lib/ share/ SRC/ var/



Installation layout


Configuring OpenNebula 2.2

Configuration Interface

$ONE_LOCATION/etc/im_*/im_*.conf

Defines monitoring probes

Match-making scheduler (default)

Placement policies configured per VM

$ONE_LOCATION/etc/oned.conf

General configuration

Defines the drivers used in the private cloud

$ONE_LOCATION/etc/tm_*/tm_*.conf

Defines action for generic storage operations

$ONE_LOCATION/etc/vmm_*/vmm_*.conf

Defaults values for the hypervisor


Configuring OpenNebula

The oned.conf file General configuration attributes

Monitoring intervals:

HOST_MONITORING_INTERVAL

VM_POLLING_INTERVAL

Global Paths

VM_DIR: Path to the VM directory in the cluster nodes.

SCRIPTS_REMOTE_DIR: to store driver actions in the cluster nodes

PORT : Port where oned will listen for xml-rpc calls

DEBUG_LEVEL

DB, configuration for the DB backend driver:

Sqlite

MySQL

VNC_BASE_PORT, for VNC port generation (BASE + ID)



The oned.conf file

#MANAGER_TIMER=30

HOST_MONITORING_INTERVAL = 60

VM_POLLING_INTERVAL = 60

#VM_DIR=/srv/cloud/one/var

SCRIPTS_REMOTE_DIR=/var/tmp/one

PORT=2633

DB = [ backend = "sqlite" ]

VNC_BASE_PORT = 5900

DEBUG_LEVEL=3



The oned.conf file

Networking

MAC_PREFIX, for MAC address generation

NETWORK_SIZE, default value

Image Repository

IMAGE_REPOSITORY_PATH, to store the images

DEFAULT_IMAGE_TYPE: OS, CDROM, DATABLOCK

DEFAULT_IMAGE_PREFIX: hd, sd, xvd, vd

MAC_PREFIX = "00:02"

NETWORK_SIZE = 254

IMAGE_REPOSITORY_PATH = /srv/cloud/images

DEFAULT_IMAGE_TYPE = "OS"

DEFAULT_DEVICE_PREFIX = "hd"



The oned.conf file

Information Drivers, to monitor cluster nodes

name: identifies the driver

executable: absolute or relative to $ONE_LOCATION/lib/mads

arguments:

hypervisor probe set (remotes dir)

Number of retries (-r)

Concurrency (-t number of threads)

IM_MAD = [

name = "im_kvm",

executable = "one_im_ssh",

arguments = “-r 0 –t 15 kvm" ]



The oned.conf file

Transfer Drivers, to interface with the storage


executable: path to driver executable

arguments: storage commands configuration file

TM_MAD = [

name = "tm_nfs",

executable = "one_tm",

arguments = "tm_nfs/tm_nfs.conf" ]



The oned.conf file

Virtualization Drivers, to interface the hypervisors


executable: absolute or relative to $ONE_LOCATION/lib/mads

arguments: same as Information Drivers

default: default values for the hypervisor

type: format of the VM description used by the driver: xen, kvm or xml

VM_MAD = [

name = "vmm_kvm",

executable = "one_vmm_ssh",

arguments = “-t 15 –r 0 kvm",

default = “vmm_ssh/vmm_ssh_kvm.conf",

type = "kvm" ]



The oned.conf file

Other sections, to interface the hypervisors

Hooks

Auth Manager

Hands on!

Check and adjust the values of oned.conf for your cloud



The oneadmin account Accounts in OpenNebula

oneadmin, has enough privileges to perform any operation on any object. It is created the first time OpenNebula is started using the ONE_AUTH data

Regular user accounts must be created by oneadmin and they can only

manage their own objects.

Configuring the oneadmin account

Environment variables: ONE_AUTH, ONE_LOCATION and

ONE_XMLRPC

$ tail .bashrc

export ONE_LOCATION=/srv/cloud/one

export ONE_AUTH=$HOME/.one/one_auth

export PATH=$PATH:$ONE_LOCATION/bin



The oneadmin account

Create the password file

Start OpenNebula using the init scripts

$ mkdir .one

$ cd .one

$ cat one_auth

oneadmin:onecloud

$ source .bashrc

$ echo $ONE_AUTH

/srv/cloud/one/.one/one_auth

$ one start

$ less $ONE_LOCATION/var/oned.log

Thu May 05 18:03:11 2011 [ONE][I]: Init OpenNebula Log system

...

Be sure to configure the oneadmin account (specially, create the ONE_AUTH file) before

starting OpenNebula for the first time.


Configuring the Private Cloud

Managing hosts and clusters

Hosts are cluster worker-nodes defined with

Hostname of the node or IP

Information Driver to be used to monitor the host

Storage Driver to clone, delete, move or copy images into the

host

Virtualization Driver to boot, stop, resume VMs in the host

Hosts are managed with the onehost utility

Create & delete hosts

List the hosts

Show detailed information from a host

Enable/Disable a host




Hands on! Register the hosts of your private cloud (front-end

will also act as a worker node)

$ onehost add pcaulaX im_kvm vmm_kvm tm_nfs

$ onehost add pcaulaY im_kvm vmm_kvm tm_nfs

$ onehost list

ID NAME CLUSTER RVM TCPU FCPU ACPU TMEM FMEM STAT

0 pcaulaX default 0 0 0 100 0K 0K on

1 pcaulaY default 0 0 0 100 0K 0K on

...

$ cat $ONE_LOCATION/var/oned.log

Mon May 2 18:06:35 2011 [InM][I]: Monitoring host pcaula7 (0)

Mon May 2 18:06:35 2011 [InM][I]: Monitoring host pcaula10 (1)

Mon May 2 18:06:38 2011 [InM][D]: Host 0 successfully monitored.

Mon May 2 18:06:39 2011 [InM][D]: Host 1 successfully monitored.

...

$ onehost list

ID NAME CLUSTER RVM TCPU FCPU ACPU TMEM FMEM STAT

0 pcaula7 default 0 200 199 200 1.9G 1.5G on

1 pcaula10 default 0 200 200 200 1.9G 1.5G on




By default, all hosts belong to the default logical cluster.

Clusters are managed using the onecluster command

Create & delete clusters

List the available clusters

Add & remove hosts from the clusters




Hands on!

Use the onehost command to view detailed information of the

hosts

Use the onehost command to enable/disable hosts

Use the onecluster command to view the clusters

Use the onecluster command to create/add hosts/remove a

cluster



Managing Users

Users are defined within OpenNebula by:

ID unique identifier for the user

Name of the user, used for authentication

Password used for authentication

Users are managed with the oneuser utility

Create & delete users

List the users in the cluster



Managing Users

Hands on!

List current users of your Cloud

Create a new user

$ oneuser create helen mypass

User “Helen” should put helen:mypass in $ONE_AUTH

$ oneuser list

UID NAME PASSWORD ENABLE

0 oneadmin c24783ba96a35464632a624d9f829136edc0175e True

2 helen 34a91f713808846ade4a71577dc7963631ebae14 True

$ oneuser delete helen



Logging and debugging information

The operations of the OpenNebula daemon and scheduler are

logged in:

oned: $ONE_LOCATION/var/oned.log, Its verbosity is set by

DEBUG_LEVEL in $ONE_LOCATION/etc/oned.conf.

Scheduler (mm_sched): All the scheduler information is collected

into the $ONE_LOCATION/var/sched.log file.

VM logs and files are in $ONE_LOCATION/var/<VM_ID>,

more in a few slides...

Drivers can activate ONE_MAD_DEBUG in the associated RC file

(or in $ONE_LOCATION/etc/defaultrc)





Installing & Configuring OpenNebula

• Installing OpenNebula 2.2

• Configuring OpenNebula 2.2

• Managing Hosts & Clusters

• Managing Users

• Logging and Debugging Information





Basic Usage of the Private Cloud

• Virtual Networks

• Images

• Virtual Machines


A Virtual Network (vnet) in OpenNebula

Defines a separated MAC/IP address space to be used by VMs

A vnet is associated with a physical network through a bridge

Virtual Networks can be isolated (at layer 2 level)

Virtual Network definition

Name, of the network

Type

Fixed, a set of IP/MAC leases

Ranged, defines a network range

Bridge, name of the physical bridge in the physical host where

the VM should connect its network interface.

Virtual Networks are managed with the onevnet utility

Virtual Networks

Overview


Virtual Networks

Example, create and manage Virtual Networks

Hands on!

Define and create two networks

$ vi red.net

NAME = "Red LAN"

TYPE = RANGED

BRIDGE = br0

NETWORK_SIZE = C

NETWORK_ADDRESS = 192.168.XX.0

$ vi blue.net

NAME = "Blue LAN"

TYPE = FIXED

BRIDGE = br0

LEASES = [IP=192.168.YY.5]

LEASES = [IP=192.168.YY.10]

LEASES = [IP=192.168.YY.15]

LEASES = [IP=192.168.YY.20]

LEASES = [IP=192.168.YY.25]


Virtual Networks

Example, create and manage Virtual Networks

Hands on!

Use the onevnet command to list and show networks

Modify the fixed network to add/remove leases with the

(addleases and rmleases option)

Leases can be public or private to the user, check and modify the

network status


Define NICs attached to a given virtual network. The VM will

get a NIC with a free MAC in the network and attached to the

bridge

Prepare the VM to use the IP. Sample scripts to set the IP

based on the MAC are provided.

Virtual Networks

Using Virtual Networks within your VMs

#A VM with two interfaces each one in a different vlan

NIC=[NETWORK="Blue LAN"]

NIC=[NETWORK="Red LAN"]

#Ask for a specific IP/MAC of the Red vlan

NIC=[NETWORK="Red LAN", IP=192.168.0.3]

IP: 10.0.1.2

MAC: 02:01:0A:00:01:02

oned.conf IP Address

IP to MAC correspondence


Images

Overview

An Image in OpenNebula’s repository

A virtual machine disk to be used as OS or DATA device.

Images can be presistent and/or public

Images modifications can be saved as another image

Image Types:

OS: contains a working operative system

CDROM: readonly data

DATABLOCK: A storage for data. Can be created either from previous

existing data, or as an empty drive.

Images are stored in the repository (/srv/cloud/images in this course)


Images

Automatic Disk Layout for Images

OS

Context ISO

CD-ROM

swap

Data 1

Data N

…

hda

hdb

hdc

hdd

hde

hdn

• Prefix (hd,sd): set as

default in oned.conf

• Can be set per image in

its template

• Target (hda…): can be

set in the VM template

for the DISK


Images

Defining a Virtual Machine Disk Image

#---------------------------------------

# Name of the Image

#---------------------------------------

NAME = "vm-example" # Mandatory

#---------------------------------------

# Image Meta-Data

#---------------------------------------

TYPE = OS | CDROM | DATABLOCK

DESCRIPTION = "of the contents of the Image"

PUBLIC = YES | NO

PERSISTENT = YES | NO

#---------------------------------------

# VM Attach attributes

#---------------------------------------

DEV_PREFIX = "to generate disk targets"

BUS = "type of device to emulate (ide,scsi,virtio)"


Images

Defining a Virtual Machine Disk Image

#---------------------------------------

# Source of the Image (use just one)

#---------------------------------------

PATH = "URL to copy the image to the repo"

SOURCE = "raw disk source (no copy)"

#---------------------------------------

# DATABLOCK generation (no path given)

#---------------------------------------

SIZE = "for the data disk in MB"

FSTYPE = "to format the image"


Images

Example, Register Images

Hands on!

Define and create two images

$ vi ttylinux.img

NAME = “ttylinux"

TYPE = OS

PUBLIC = yes

DESCRIPTION = “ttylinux with context. Root passwd is password“

PATH = <put_the_path_here>

PERSISTENT = no

$ vi data.img (bug in 2.2, include /sbin in PATH)

NAME = “data"

TYPE = DATABLOCK

DESCRIPTION = “user data”

PUBLIC = no

PERSISTENT = yes

SIZE = 100

FSTYPE = ext2


Images

Example, Register Images

Hands on!

Check images with oneimage list and show

Change public and persistent attributes

Check the contents of the repository (/srv/cloud/images)


Images

Using Images with your Virtual Machines

Define DISKs attached to the virtual machine.

Select the image by name or id (IMAGE_ID preferred)

Overwrite attributes if needed (TARGET, BUS)

Prepare the VM to use the disk layout to ease usage

# OS image, mapped to sda.

DISK = [ IMAGE = "Debian 5.0" ]

# First DATABLOCK image, mapped to sde

DISK = [ IMAGE_ID = 4 ]

# swap, sdd

DISK = [ TYPE = swap, SIZE = 1024, READONLY = "no" ]


Virtual Machines

Overview

A Virtual Machine in OpenNebula

A capacity in terms memory and CPU

A set of NICs attached to one or more virtual networks

A set of disk images, to be “transfered” to/from the execution host.

A state file (optional) or recovery file, with the memory image of a

running VM plus some hypervisor specific information.

Virutal Machines are defined in a VM template

Each VM has an unique ID in OpenNebula the VMID

All the files (logs, images, state files...) are stored in

$ONE_LOCATION/var/<VMID>


Virtual Machines

Virtual Machine Template# Name of the VM

NAME = "vm-example" # Optional, Default: one-$VMID

# Capacity

CPU = "amount_of_requested_CPU"

MEMORY = "amount_of_requested_MEM"

VCPU = "number of virtual cpus"

# OS and boot options

OS = [

kernel = "path_to_os_kernel", # para-virtualization

initrd = "path_to_initrd_image", # para-virtualization

kernel_cmd = "kernel_command_line",

root = "device to be mounted as root"

bootloader = "path to the boot loader exec”

boot = "device to boot from" ]

# Features of the hypervisor

FEATURES = [

pae = "yes|no", # Optional, KVM

acpi = "yes|no" ] # Optional, KVM


Virtual Machines

Virtual Machine Template

# VM Disks

# Using the Image Repository

DISK = [

image = "name of the image (deprecated)",

image_id = "id of the image",

bus = "override image attribute”,

target = "override default layout",

driver = “override image attribute" ]

# Using a source URL

DISK = [

type = "floppy|disk|cdrom|swap|fs|block",

source = "path_to_disk_image_file|physical_dev",

format = “type for fs disks”,

size = "size_in_GB",

target = "device_to_map_disk",

bus = "ide|scsi|virtio|xen",

readonly = "yes|no",

clone = "yes|no",

save = "yes|no" ]


Virtual Machines


# Network Interfaces

NIC = [

network = "name_of_the_virtual_network",

ip = "ip_address",

bridge = "name_of_bridge_to_bind_if",

target = "device_name_to_map_if",

mac = "HW_address",

script = "path_to_script_to_bring_up_if",

Model = "NIC model"]

# I/O Interfaces

INPUT = [

type = "mouse|tablet",

bus = "usb|ps2|xen" ]


Virtual Machines


# I/O Interfaces

GRAPHICS = [

type = "vnc|sdl",

listen = "IP-to-listen-on",

port = "port_for_VNC_server",

passwd = "password_for_VNC_server" ]

# Raw Hypervisor attributes

RAW = [

type = "xen|kvm",

data = "raw_domain_configutarion"]

Not all the parameters are supported for each hypervisor. Complete

reference and examples for all sections in

http://opennebula.org/documentation:rel2.2:template


Virtual Machines

Example, define a simple VM

Hands on!, create a simple VM

Use the ttylinux image

Use the Red network

Enable VNC access to monitor the boot process

NAME = ttylinux

CPU = 0.1

MEMORY = 64

DISK = [ IMAGE_ID = 0 ]

NIC = [ NETWORK_ID = 0 ]

FEATURES = [ acpi="no" ]

GRAPHICS = [ type="vnc", listen="0.0.0.0", keymap="es" ]


Virtual Machines

Example, define a simple VM

Hands on!

Check the progress of the VM with onevm top

Check the log with $ONE_LOCATION/var/0/vm.log

Check that the image boot with a vnc client

onevm command options:

Operations: create, deploy shutdown, livemigrate, stop, cancel, resume, suspend, delete, restart

Information: list, show, top, history


Virtual Machines

Life-cycle of a VM (simplified)


Virtual Machines

Example, manage a simple VM

Hands on!

Check status of the vnets and images in use by the VM

Stop/Resume the Virtual Machine, check VM directory

Migrate the Virtual Machine (cold migration)

Live Migrate the VM

Update the QEMU protocol to “qemu+ssh” in $ONE_LOCATION/var/remotes/kvm/kvmrc

onehost sync (wait to monitor) – check /var/tmp/one

Create another VM and check connectivity

Add another disk with the datablock (use TARGET hdc, qemu IDE

limitation)


Virtual Machines

Example, manage a Simple VM Hands on!

Enable network access by adding a NIC to Red and Blue networks (no needed with VNC…)

Test ssh, ping and VM connectivity

Add a tap interface to the physical host and put it un “Red LAN”

# apt-get install openvpn

# openvpn --mktun --dev tap0

# ifconfig tap0 192.168.XX.50/24 up

# brctl addif br0 tap0

# route del -net 192.168.XX.0/24 tap0

# route add -net 192.168.XX.0/24 br0


Virtual Machines

Guidelines to Prepare a Virtual Machine You can use any VM prepared for the target hypervisor

Hint I: Place the vmcontext.sh script in the boot process to make

better use of VLANs

Hint II: Do not pack useless information in the VM images:

swap. OpenNebula can create swap partitions on-the-fly in the

target host

Scratch or volatile storage. OpenNebula can create plain FS on-

the-fly in the target host

Hint III: Install once and deploy many; prepare master images

Hint IV: Use the Image Repository and default layout

Hint V: Do not put private information (e.g. ssh keys) in the master

images, use the CONTEXT

Hint VI: Pass arbitrary data to a master image using CONTEXT





Basic Usage of the Private Cloud

• Virtual Networks

• Images

• Virtual Machines





Advance Usage of the Private Cloud

• Context for Virtual Machines

• Scheduling Virtual Machines

• Sunstone


Virtual Machine Context

Overview

• Block device (ISO9660) with configuration data needed at boot

time

• Information includes variables and arbitrary files

• VM should be prepared to make use of context (mount + read)



Overview

• Context is defined in the VM template

#---------------------------------------

# Context for the VM

# values can be:

# $<template_variable>

# $<template_variable>[<attribute>]

# $<template_variable>[<attribute>, <attribute2>=<value2>]

# $<vm_id>.<context_var>

#---------------------------------------

CONTEXT = [

var_1 = "value_1",#In context.sh as var_1=”val_1” (sh syntax)

var_n = "value_n",#In context.sh as var_N=”val_N” (sh syntax)

files = "space-separated list of paths to include in context dev",

target= "device to attach the context device" ]



Example, create a Virtual Machine with Context

Hands on!

Check the boot process of ttylinux (rc.local and vmcontext)

Mount context cd-rom

Source context.sh

Execute target initialization script

CONTEXT = [

files = “<path_to>/init.sh /srv/cloud/one/.ssh/id_rsa.pub",

root_pubkey = "id_rsa.pub"

]

$ more init.sh

#!/bin/bash

. /mnt/context/context.sh

if [ -f /mnt/context/$ROOT_PUBKEY ]; then

cat /mnt/context/$ROOT_PUBKEY >> /root/.ssh/authorized_keys

fi



Example, create a Virtual Machine with Context

Hands on!

Create and define a VM with context

Study and modify init.sh to set up hostname

Check password-less ssh with id_rsa.pub


Scheduling Virtual Machines

Placement constraints

Tunning the placement of VMs with the Match-making

scheduler

First those hosts that do not meet the VM requirements are

filtered out (REQUIREMENTS)

RANK is evaluated for the remaining hosts

That with the highest RANK is used for the VM

Placement policies are specified per VM

#---------------------------------------

# Scheduler

#---------------------------------------

# Use Host Monitor attributes

REQUIREMENTS = "Bool_expression_for_reqs"

RANK = "Arith_expression_to_rank_hosts"



Sample Placement Heuristics Packing (Minimize the number of cluster nodes in use)

Heuristic: Pack the VMs in the cluster nodes to reduce fragmentation

Implementation: Use those nodes with more VMs running first ( RANK

= RUNNING_VMS )

Striping Policy (Maximize the resources available to VMs)

Heuristic: Spread the VMs in the cluster nodes

Implementation: Use those nodes with less VMs running first (RANK =

"- RUNNING_VMS“)

Load-aware Policy (Maximize resources)

Heuristic: Use those nodes with less load

Implementation: Use those nodes with more FREECPU first (RANK =

FREECPU)



Example, guide the scheduling of the VMs Hands on!

Try VM pinning (choose a variable from onehost show) -

REQUIREMENTS

Experiment with the previous policies - RANK


Sunstone

Overview Web application to perform admin tasks

Sunstone is not a public cloud user tool

Server must have access to the XML-RPC API


Sunstone

Installation & Configuration Install ruby gems needed by the serve

Add /var/lib/gems/1.8/bin to PATH

Start the server as oneadmin

-H hostname for the server

-p port

Log information in $ONE_LOCATION/var/sunstone.log

# apt-get install libopenssl-ruby

# gem install json sinatra thin rack

$ sunstoner-server –H pcaulaXX.cesga.es start


Sunstone

Example, use the GUI Hands on!

Manage the cloud (hosts, vnets, images, vms…) through

sunstone





Advance Usage of the Private Cloud

• Context for Virtual Machines

• Scheduling Virtual Machines

• Sunstone





Hybrid Cloud Computing

• Hybrid Cloud Computing

• Installing a Hybrid Cloud with EC2

• Configuring the Hybrid Cloud

• Using the OpenNebula-EC2 Cloud



Overview

External Clouds are like any other host

Placement constraints

OpenNebula distribution includes EC2 drivers

VMs can be local or remote

VM connectivity has to be configured, usually VPNs


Installing the Hybrid Cloud

Runtime Requirements (front-end)

EC2 libraries and tools.

EC2 tools credentials:

Add those variables to .bashrc and test the tools

# apt-get install ec2-ami-tools ec2-api-tools

$ export EC2_PRIVATE_KEY=/srv/cloud/one/ec2/pk.pem

$ export EC2_CERT=/srv/cloud/one/ec2/cert.pem

$ ec2-describe-images

IMAGE ami-0742a66e /rubensm-

amis.s3.amazonaws.com/image.manifest.xml 418314910487

available private i386 machine

IMAGE ami-e142a688 rubensm-

amis.s3.amazonaws.com/image.manifest.xml 418314910487

available private i386 machine



OpenNebula drivers for EC2

Configure OpenNebula to use the EC2 drivers

IM_MAD = [

name = "im_ec2",

executable = "one_im_ec2",

arguments = "im_ec2/im_ec2.conf" ] # No. of instances of each type

VM_MAD = [

name = "vmm_ec2",

executable = "one_vmm_ec2",

arguments = "vmm_ec2/vmm_ec2.conf", # Defaults, e.g. keypair

type = "xml" ]

TM_MAD = [ #No actual transfers are made by OpenNebula to EC2

name = "tm_dummy",

executable = "one_tm",

arguments = "tm_dummy/tm_dummy.conf" ]




Configure the OpenNebula account (will use bashrc)

$ vim $ONE_LOCATION/etc/vmm_ec2/vmm_ec2rc

#--------------------------------------------------------------------

# EC2 API TOOLS Configuration.

#--------------------------------------------------------------------

EC2_HOME=“/usr/”

#EC2_PRIVATE_KEY="/srv/cloud/one/ec2/certs/pk.pem"

#EC2_CERT="/srv/cloud/one/ec2/certs/cert.pem"

Configure the capacity to be outsourced

$ vim $ONE_LOCATION/etc/im_ec2/im_ec2.conf

#-------------------------------------------------------------------

# Max number of instances that can be launched into EC2

#--------------------------------------------------------------------

SMALL_INSTANCES=5

LARGE_INSTANCES=

EXTRALARGE_INSTANCES=




Amazon EC2 cloud is managed by OpenNebula as any other cluster

node. Restart the oned, and check that the new drivers are loaded

$ one stop; one start

$ more $ONE_LOCATION/var/oned.log

Fri Jan 15 18:16:46 2010 [VMM][I]: Loading Virtual Machine Manager

driv

Fri Jan 15 18:16:46 2010 [VMM][I]: Loading driver: vmm_kvm (KVM)

Fri Jan 15 18:16:47 2010 [VMM][I]: Driver vmm_kvm loaded.

Fri Jan 15 18:16:47 2010 [VMM][I]: Loading driver: vmm_ec2 (XML)

Fri Jan 15 00:16:47 2010 [InM][I]: Loading Information Manager

drivers.

Fri Jan 15 00:16:47 2010 [InM][I]: Loading driver: im_kvm

Fri Jan 15 00:16:47 2010 [InM][I]: Driver im_kvm loaded

Fri Jan 15 00:16:47 2010 [InM][I]: Loading driver: im_ec2


Configuring the Hybrid Cloud

Register the EC2 Cloud

Hands on!


Check the information and characteristics of the new host

$ onehost create ec2 im_ec2 vmm_ec2 tm_dummy

$ onehost list

ID NAME RVM TCPU FCPU ACPU TMEM FMEM STAT

0 host01 0 200 200 200 2017004 1667080 on

1 host02 1 200 200 200 2017004 1681676 on

2 ec2 0 500 500 500 8912896 8912896 on



Using EC2 zones and multiple accounts You can use several accounts by adding a driver for each account

(use the arguments attribute, -k and –c options). Create a host that

uses the driver

You can use multiple EC2 zones, add a driver for each zone (use the

arguments attribute, -u option). Create a host that uses the driver

VM_MAD = [

name = "vmm_ec2",


arguments = "vmm_ec2/vmm_ec2.conf –k /srv/cloud/...",

type = "xml" ]

VM_MAD = [

name = "vmm_ec2",


arguments = "vmm_ec2/vmm_ec2.conf –u http://...",

type = "xml" ]


Using the Hybrid Cloud

Defining an EC2 Virtual Machine

Virtual Machines can be instantiated locally or in EC2

The template must provide a description for both instantiation

methods.

The EC2 counterpart of your VM (AMI_ID) must be available for

the driver account

The EC2 VM template attribute:

EC2 = [

AMI = "ami_id for this VM",

KEYPAIR = "the keypair to use the instance",

AUTHORIZED_PORTS = "ports to access the instance",

INSTANCETYPE = "m1.small...",

ELASTICIP = "the elastic ip for this instance",

CLOUD = "host (EC2 cloud) to use this description with"

]



Example, Use the OpenNebula –Ec2 Hybrid Cloud

Hands on!

Add an EC2 counterpart to the ttylinux image

$ vi ttylinux.one

#EC2 template machine, this will be use if this VM is created in EC2

EC2 = [ AMI="ami-ccf405a5",

KEYPAIR="keypair",

AUTHORIZED_PORTS="22",

INSTANCETYPE=m1.small]

#Add this if you want to use only EC2 cloud

REQUIREMENTS = "HOSTNAME = \"ec2\""



Example, Use the OpenNebula –Ec2 Hybrid Cloud

Hands on!

Check progress

Check VM information with onevm show and Sunstone

$ onevm list

ID USER NAME STAT CPU MEM HOSTNAME TIME

5 oneadmin ttylinux runn 0 0K ec2 00 00:00:59

$ ec2-describe-instances

RESERVATION r-53599b3f 418314910487 default

INSTANCE i-a884b7c7 ami-ccf405a5 ec2-50-19-44-

30.compute-1.amazonaws.com ip-10-85-65-203.ec2.internal running

keypair 0 m1.small 2011-05-03T01:14:08+0000

us-east-1d aki-407d9529 monitoring-disabled

50.19.44.30 10.85.65.203 ebs

BLOCKDEVICE /dev/sda1 vol-ed935186 2011-05-

03T01:14:31.000Z



Example, Use the OpenNebula –Ec2 Hybrid Cloud$ onevm show 17

...

VIRTUAL MACHINE TEMPLATE

CPU=0.5

...

EC2=[

AMI=ami-ccf405a5,

KEYPAIR=keypair ]

IP=ec2-50-19-44-30.compute-1.amazonaws.com

...

$ ssh -i keypair.pem [email protected]

Linux ip-10-212-134-128 2.6.21.7-2.fc8xen-ec2-v1.0 #2 SMP Tue Sep 1

10:04:29 EDT 2009 i686

…

ubuntu@ip-10-85-65-203:~$ exit

This costs money!

$ onevm shutdown 17

$ onehost disable ec2

$ onehost list






• Hybrid Cloud Computing

• Installing a Hybrid Cloud with EC2

• Configuring the Hybrid Cloud

• Using the OpenNebula-EC2 Cloud





Public Cloud Computing

• Public Cloud Computing with OpenNebula

• Installing a Public Cloud with EC2 API

• Configuring the Public Cloud

• Using the Public Cloud (EC2)


Public Cloud Computing with OpenNebula

Overview You can use multiple interfaces for the Cloud

Transparent to your setup:

Hypervisor

Storage Model

Hybrid configuration

Supports HTTP and HTTPS protocols

EC2 authentication based on OpenNebula credentials

Public Cloud users need an OpenNebula account

Client tools uses EC2 libraries

Potential integration with EC2 tools (EC2_URL problems for example)

Shipped with OpenNebula

Includes a simple S3 replacement


Installing the Public Cloud

Runtime Requirements (front-end)

OpenNebula distribution supports two Cloud interfaces: the EC2

Query API and OCCI

Additional requirements: EC2 development library, web server and

web framework

# gem install amazon-ec2 uuid

# gem install sequel

# apt-get install curl libcurl3 libcurl4-gnutls-dev

# gem install curb

# gem install sqlite3-ruby

Add a “FQDN” for our Public Cloud

# vim /etc/hosts

127.0.0.1 localhost

#127.0.1.1 pcaulaX

193.144.33.y pcaulaX pcaulaX.opennebula.org


Configuring the Public Cloud

Server Options and Instance types

The EC2 service is configured in $ONE_LOCATION/etc/econe.conf

Hands on!

Study the configuration file and adjust it to your cloud

# OpenNebula sever contact information

ONE_XMLRPC=http://localhost:2633/RPC2

# Host and port where econe server will run

SERVER=pcaulaX.opennebula.org

PORT=4567

# SSL proxy that serves the API (set if is being used)

#SSL_SERVER=fqdm.of.the.server

# VM types allowed and its template file (inside templates directory)

VM_TYPE=[NAME=m1.small, TEMPLATE=m1.small.erb]



Define the Instances

You have to define the correspondence between types (simple) and

local instantiation of VMs (hard, you should be fine by now)

Capacity allocated by this VM type (CPU, MEMORY)

Your cloud requirements, e.g. force to use a given kernel (OS) or place

public VMs in a given set of cluster nodes (REQUIREMENTS)

The network used by Public VMs (NIC)

VM Types are defined in econe.conf. Templates for the VM templates

are in $ONE_LOCATION/etc/ec2query_templates

Templates for VM Types are erb files <% Ruby code here %>, you

should not need to modify that.



Define the Instances

$ more m1.small.erb

NAME = eco-vm

#Adjust Capacity for this instance type

CPU = 0.1

MEMORY = 64

DISK = [ IMAGE_ID = <%= erb_vm_info[:img_id] %> ]

NIC = [ NETWORK_ID = 0 ]

IMAGE_ID = <%= erb_vm_info[:ec2_img_id] %>

INSTANCE_TYPE = <%= erb_vm_info[:instance_type ]%>

<% if erb_vm_info[:user_data] %>

CONTEXT = [

EC2_USER_DATA="<%= erb_vm_info[:user_data] %>",

TARGET="hdc” ]

<% end %>



Start the EC2 Server Hands on!

Start the EC2 server

Adjust the m1.small template

Create additional “public” users with oneuser create

$ econe-server start

$ /usr/sbin/lsof -Pi

Check $ONE_LOCATION/var/econe-server.log for errors


Using the Public Cloud

The econe Toolset

The econe-tools are a subset of the functionality provided by the

onevm utility, and resembles the ec2-* cli

EC2 ecosystem can be used (e.g. elasticfox, euca2ools…)

Image related commands are:

econe-upload, place an image in the Cloud repo and returns ID

econe-describe-images, lists the images

econe-register, register an image

Instance related commands are:

econe-run-instances, starts a VM using an image ID

econe-describe-instances, lists the VMs

econe-terminate-instances, shutdowns a VM



The econe Toolset

User authentication is based in the OpenNebula credentials

AWSAccessKeyId is OpenNebula's username

AWSSecretAccessKey is OpenNebula's password

Pass your credentials to the econe-tools by (in this order)

Command arguments (-K <username>, -S <pass>)

Environment EC2_ACCESS_KEY and EC2_SECRET_KEY

Environment ONE_AUTH

Point econe-tools to your target cloud

Command arguments (-U <http|https>://<fqdn>:<port>) port

needed if not the default for the protocol

EC2_URL environment



Example, Running a VM through the EC2 Interface

Hands on!

Check the images in your cloud and start using it

Compare the econe-* (public view) and one* and sunstone

(local view) evolution and information

Check the template build by the econe server (onevm show)

Upload the ttylinux image again and instance it



Example, Running a VM through the EC2 Interface$ econe-upload -U http://node-x.opennebula.org:4567 --access-key ec2-

user --secret-key ec2-pass /srv/cloud/images/ttylinux/ttylinux.img

Success: ImageId ami-00000003

$ export EC2_URL=http://pcaulax.opennebula.org:4568

$ export EC2_ACCESS_KEY=ec2-user

$ export EC2_SECRET_KEY=ec2-pass

$ econe-describe-images -HOwner ImageId Status Visibility Location

-----------------------------------------------------------------------

ec2-user ami-00000003 available private 23151fac850e5...

This is the local view not accessible to public cloud users

$ oneimage listID NAME TYPE REGTIME PUB PER STAT #VMS

...

3 ec2-user ec2-71654e30-0872-01 OS Jan 22, 2011 No No rdy 0

$ oneimage show 3




$ econe-run-instances ami-00000003ec2-user ami-00000004 i-16 m1.small

$ econe-describe-instances -HOwner Id ImageId State IP Type

------------------------------------------------------------------------------

ec2-user i-10 ami-00000003 running 172.16.10.7 m1.small

This is the local view not accessible to public cloud users

$ onevm list

ID USER NAME STAT CPU MEM HOSTNAME TIME

1 oneuser ttylinux runn 99 63.5M n04 01 02:41:14

10 ec2-user eco-vm runn 99 63.8M n04 00 01:05:28

$ onevm show 14

VIRTUAL MACHINE 14 INFORMATION

ID : 14

NAME : eco-vm

STATE : ACTIVE

...



SSL Security to access the EC2 Server

SSL security is handle by a proxy that forwards the request to

the EC2 Query Service and takes back the answer to the client

Requirements:

A server certificate for the SSL connections

An HTTP proxy that understands SSL

EC2Query Service configuration to accept petitions from the proxy




Hands on!

Install the proxy (lighttpd in our course)

Generate the server certificates for your cloud

Configure the proxy

Restart the services and test the new SSL enabled Cloud

(https://pcaulaX.cesga.es:443)

# apt-get install lighttpd

# apt-get install ssl-cert

# /usr/sbin/make-ssl-cert generate-default-snakeoil

# cat /etc/ssl/private/ssl-cert-snakeoil.key /etc/ssl/certs/ssl-cert-

snakeoil.pem > /etc/lighttpd/server.pem



SSL Security to access the EC2 Server# vim /etc/lighttpd/lighttpd.conf

server.modules = (

...

"mod_compress",

"mod_proxy"

...

## bind to port (default: 80)

server.port = 443

...

#### proxy module

proxy.server = ( "" =>

("" =>

(

"host" => "127.0.0.1",

"port" => 4567

)

)

)

#### SSL engine

ssl.engine = "enable"

ssl.pemfile = "/etc/lighttpd/server.pem"




$ vim /srv/cloud/one/etc/econe.conf

#SERVER=node-15.opennebula.org

SERVER = 127.0.0.1

PORT=4568

# SSL proxy that serves the API (set if is being used)

SSL_SERVER = pcaulaX.opennebula.org

$ econe-server stop

$ econe-server start

# service lighttpd restart

$ econe-describe-instances -K oneadmin -S onecloud -U

https://pcaula7.cesga.es:443

oneadmin i-6 0 running

192.168.169.1 m1.small





Public Cloud Computing

• Public Cloud Computing with OpenNebula

• Installing a Public Cloud with EC2 API

• Configuring the Public Cloud

• Using the Public Cloud (EC2)





Customizing your Cloud

• Adapting & Customizing OpenNebula

• The Storage Subsystem

• The Information Subsystem

• Using Hooks

• Fault Tolerance

• Network Isolation

• More Customization


Customizing & Extending OpenNebula

Overview You can customize your cloud by:

Tunning or adapting the transfer operations to your storage back-end

Adding new monitorization probes to improve the VM placement

Adjusting VM operations to your hypervisor installation

Trigger custom actions on specific VM events (e.g. “on VM creation update the accounting DB” or “on VM shutdown send an email”)

You can extend your cloud by:

Developing new drivers for other hypervisors

Developing new drivers for other storage back-ends

Developing Cloud applications using the OpenNebula API or the Cloud APIs

OpenNebula is very scripting friendly, drivers can be written in any language. You can modify thecurrent ones or use them as templates for new ones.


The Storage Backend

OverviewOpenNebula issue generic storage operations (check var/<vm_id>/transfer.*)

Transfer Driver process the core script

Action programs interpret the semantics of generic actions depending on the storage back.-end


The Storage Backend

Transfer Manager Operations

OpenNebula requests the following abstract operations over a VM image

CLONE: This action will basically make a copy of the image from ORIGIN to DESTINATION.

LN: Creates a symbolic link in DESTINATION that points to ORIGIN

MKSWAP: Generates a swap image in DESTINATION. The size is given in ORIGIN in MB.

MKIMAGE: Creates a disk image in DESTINATION and populates it with the files inside ORIGIN directory.

DELETE: Deletes ORIGIN file or directory.

MV: Moves ORIGIN to DESTINATION.


The Storage Backend

The Transfer Manager Action Scripts

Actions are defined in

$ONE_LOCATION/etc/tm_<storage>/tm_<storage>.conf

$ more /srv/cloud/one/etc/tm_ssh/tm_ssh.conf

CLONE = ssh/tm_clone.sh

LN = ssh/tm_ln.sh

MKSWAP = ssh/tm_mkswap.sh

MKIMAGE = ssh/tm_mkimage.sh

DELETE = ssh/tm_delete.sh

MV = ssh/tm_mv.sh

Actions scripts are placed in

$ONE_LOCATION/lib/tm_commands/<storage>/

$ ls /srv/cloud/one/lib/tm_commands/ssh/

tm_clone.sh tm_delete.sh tm_mkimage.sh tm_mv.sh

tm_context.sh tm_ln.sh tm_mkswap.sh


Example, Customize your storageThe Storage Backend

Hands on!

Study the tm_clone.sh script

Check the semantics of Storage operations for the ssh backend

Delete and create a host using ssh backend

Example 1: Make swap partitions in local storage

Take a look to tm_mkswap from ssh

Link the swap partition to original DST path

Example 2: Make the clone script aware of compressed images

(*.gz)


The Storage Backend

Example, Customize your storage

$ cat $ONE_LOCATION/lib/tm_commands/ssh/tm_clone.sh

. $TMCOMMON

...

log "Creating directory $DST_DIR"

exec_and_log "ssh $DST_HOST mkdir -p $DST_DIR"

...

case $SRC in

http://*)

log "Downloading $SRC"

exec_and_log "ssh $DST_HOST wget -O $DST_PATH $SRC"

;;

*)

log "Cloning $SRC"

exec_and_log "scp $SRC $DST"

;;

esac

exec_and_log "ssh $DST_HOST chmod a+w $DST_PATH"


The Information System

Overview

OpenNebula gets host information by executing an arbitrary

number of probes:

Program that returns a monitor metric (METRIC_NAME = VALUE)

Placed in $ONE_LOCATION/var/remotes/im/<hypervisor>.d

Monitor probes can be executed:

Remotely using the one_im_ssh driver

Locally to integrate it with an external monitor system (Ganglia,

Nagios…)

Remote probes are cached at the cluster nodes and sync with

onehost sync command (next time the host is monitored)

Probe information is mainly used for VM placement


The Information System

Example, customize the Monitor Information

Hands on!

Study & execute (run_probes kvm) the probes for kvm at

$ONE_LOCATION/var/remotes/im/kvm.d

Create a new monitor probe:

Compute the number of running VMS (e.g. MY_RVMS), use

virsh list, pgrep kvm…

Sync the cluster nodes

Use the new metric for VM placement


Hooks

Overview

Hooks are custom programs executed: Locally (front-end) or remotely (target host)

Upon VM related events or Host related events

OpenNebula includes:

Fault Tolerance Hooks, to restart VMs when a host crashes or restart VMs when it fails

Network Isolation Hooks, to setup VLANs

Hooks are defined in oned.conf

Name, of the hook

command, to be executed

arguments

on, event that triggers the hook

remote


Hooks: Fault Tolerance

Configuring Fault Tolerance Hooks

Host Hook (on ERROR)

resubmit (-r) or delete (-d) VMs in the host

Including suspended (y) VMs or not (n)

HOST_HOOK = [

name = "error",

on = "ERROR",

command = "host_error.rb",

arguments = "$HID -r n",

remote = no ]

VM Hook (on FAILURE)

VM_HOOK = [

name = "on_failure_resubmit",

on = "FAILURE",

command = "onevm resubmit",

arguments = "$VMID" ]


Hooks: Network Isolation

Overview IN: Only Ethernet frames from a MAC in Red LAN

OUT: Only Ethernet frames from the MAC assigned by OpenNebula

Networks are isolated at layer 2

You can put any TCP/IP service as part of the VMs (e.g. DHCP, nagios...)




Requirements (this has to be done in all the cluster nodes)

Check that ebtables package is installed

Allow oneadmin to use the ebtables command through sudo

Configure the hooks for OpenNebula

#visudo

...

oneadmin ALL=(ALL) NOPASSWD: /sbin/ebtables *

...

VM_HOOK = [

name = "ebtables-start",

on = "running",

command = "/srv/cloud/one/share/hooks/ebtables-kvm",

arguments = "one-$VMID",

remote = "yes" ]

VM_HOOK = [

name = "ebtables-flush",

…




Apply patch for OpenNebula 2.2 bug in share/hooks/ebtables-

kvm

80 if interfaces.values.flatten.include? tap

Hands on!

Start a couple of VMs in Networks Red and Blue.

Check the ebtables rules in the hosts

Check connectivity between VMs

Change the network mask of the VMs and check connectivity

Shutdown and check the ebtables rules


More Customization

Other Components that can be adapted

Authorization & Authentication, can be performed with external

drivers

ssh-keys based authentication

ldap based authentication

DB Backend, use MySQL for more performing setups

Use Ganglia, for better scalability

Develop your own components/applications (PaaS,SaaS)

XML-RPC interface

OCA (Ruby, JAVA and Python – contributed - bindings)

EC2 or OCCI APIs





Customizing your Cloud

• Adapting & Customizing OpenNebula

• The Storage Subsystem

• The Information Subsystem

• Using Hooks

• Fault Tolerance

• Network Isolation

• More Customization

Ruben S.Montero

dsa-research.org | OpenNebula.org

Distributed Systems Architecture Research Group

Universidad Complutense de Madrid

Centro de Supercomputación de Galicia

May 2011, Spain




• Describe the benefits and characteristics of virtual

infrastructures and IaaS clouds

• Describe the characteristics and architecture of the different

clouds that can be deployed with OpenNebula 2.2

• Plan and architect a private cloud

• Design, Use and Manage Virtual infrastructures (cloud

applications)

• Build public and hybrid clouds

• Adapt OpenNebula 2.2 to your datacenter


Summary, You should by able by now to ….

Building Clouds with OpenNebula2.2

Technology

Transcript of Building Clouds with OpenNebula2.2