Abdullah&Hiatham 05/02/20111. Introduction History of BoxGrinder How does Boxgrinder work ?...
-
date post
22-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Abdullah&Hiatham 05/02/20111. Introduction History of BoxGrinder How does Boxgrinder work ?...
BoxGrinder and OpenStack
CS526 Semester Project Presentation
Abdullah Sheneamer and Hiatham Ennajah
Department of Computer ScienceUniversity of Colorado at Colorado Springs
Abdullah&Hiatham 05/02/2011 1
• Introduction
• History of BoxGrinder
• How does Boxgrinder work ?
• Installation and Configuration of BoxGrinder
• BoxGrinder of Plugins
• OpenStack
• References
Outline of the Talk
Abdullah&Hiatham 05/02/2011 2
Introduction
• BoxGrinder is an open source suite for building virtual appliances (also known as virtual machines) that are ready to run on various virtualization and Cloud providers or another definition that it is a set of tools used for building appliances (virtual machines) on various platforms (KVM, Xen, VMware, EC2).
• It supports many platforms such as EC2, Xen, KVM and VMware.
• BoxGrinder is able to create appliances based on Fedora, Red Hat Enterprise Linux or CentOS, and is easy extendable to meet any requirements.
Abdullah&Hiatham 05/02/2011 3
Introduction (Cont.)
• KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V).
• Xen is a virtual-machine monitor for IA-32, x86-64, Itanium and ARM architectures. It allows several guest operating systems to execute on the same computer hardware concurrently
• VMware's desktop software runs on Microsoft Windows, Linux, and Mac OS X.
• Appliance definition files are written in YAML
Abdullah&Hiatham 05/02/2011 4
History
• History:
- Who:
-Marek Goldmann
- When:
-BoxGrinder Build 0.6.5 in 04 December 2010.
-BoxGrinder Build 0.7.0 in 19 December 2010
-BoxGrinder Build 0.7.1 in 03 January 2011
-BoxGrinder Build 0.8.0 in 25 January 2011
-BoxGrinder Build 0.8.1 released 22 February 2011
-BoxGrinder Build 0.9.0 Released 09 march 2011
-BoxGrinder Build 0.9.1 in 28 April 2011
Abdullah&Hiatham 05/02/2011 5
How does BoxGrinder work?
• BoxGrinder is creating appliances (images/virtual machines) from simple plain text Appliance Definition Files.
• There are only two simple steps to create an appliance:
1-Create Appliance Definition File
2-Run BoxGrinder
Appliance Definition
BoxGrinder Appliance
Abdullah&Hiatham 05/02/2011 6
How does BoxGrinder work? (Cont.)
• BoxGrinder consists of three projects:
1- BoxGrinder Build - a fast and easy to use command line
tool that is responsible for actually building the appliances.
2- BoxGrinder REST - a server with farm of builder nodes.
Build tasks are distributed to nodes where BoxGrinder
Build is used to build the appliance.The resulting appliance is transferred to the configured destination.
3- BoxGrinder Studio - a web front-end to BoxGrinder
REST designed to provide a friendly graphical user
experience on top of BoxGrinder REST.(Still working on it) BOXGRINDER BUILD
BOXGRINDER REST
BOXGRINDER STUDIO
Abdullah&Hiatham 05/02/2011 7
Installation and Configuration of BoxGrinder
• There are three phases to create boxgrinder:
1- Preparing environment to start building appliances.
- Fedora 13 and 14 doesn’t require preparing anything before installing BoxGrinder – use BoxGrinder RPMs – this will pull all required packages.
2- Installation. - Before you install BoxGrinder make sure you have prepared your
environment or use meta appliance
3- Building appliances
- Before you start make sure you have installed BoxGrinder Build correctly.
Abdullah&Hiatham 05/02/2011 8
Installation
• The easiest way to obtain BoxGrinder Build is to simply add BoxGrinder repo to your environment and use yum.
- You have to execute this command to install BoxGrinder in Fedora.
- “ yum install rubygem-boxgrinder-build” Command
- Other Systems use this command to install BoxGrinder on it.
- “gem install boxgrinder-build “ Command
Abdullah&Hiatham 05/02/2011 9
Building appliances
• Create appliances directory such as “ mkdir appliances “ command.
- The creating is option.• Simple Example:
name (required)This is a name for your appliance. It must be unique in all of your appliances
summaryA small (one sentence or such) summary of the appliance; purpose, goals, etc.
osThis section specifies operating system information for the appliance.Members: name – OS name, default: fedora, version – OS version; this could be a number or string: 1, rawhide, default: 12. password – this is the root password, default: boxgrinder.
hardwareThis section gives the ability to specify virtual hardware requirements.Members: cpus – virtual CPU count, default: 1, memory – memory quantity in MB, default: 256, partitions – partitioning scheme with root mounts and size in GB
packagesThis section contains all of the packages you wish to add to your appliance.
Repos: you can specify additional repositories in order to add packages to your appliance that are sourced from these repositories.
Abdullah&Hiatham 05/02/2011 10
Boxgrinder Plugins
• BoxGrinder Build has a plugin architecture. We can distinguish three types:
- Operating System plugins – generating base appliance for
selected OS.
- Platform plugins – creating converted base appliance for
selected platform.
- Delivery plugins – designed to deliver your new appliance to a specfied location. For example as a tar file to a remote server or just reqister as AMI on EC2.
Appliance Definition
BoxGriinder
Repository
OS Plugin
Platform Plugin
Delivery Plugin
Base Appliance
Converted Appliance
Appliance
Abdullah&Hiatham 05/02/2011 11
• Operating system plugins:
• The goal of this kind of plugin is to create a base image for the selected operating system. Each plugin must inherit the BaseOperatingSystemPlugin class:
• Supported Operating Systems:
Fedora plugin
CentOS plugin
RHEL plugin
• Fedora Operating System Plugin Configuration:
• The configuration of the plugin would have the following three declarations at the begigning of the file:
• Plugins: # Plugin name.
• fedora: # OS supported.
• format: # Disk format to use. Default: raw.
BoxGrinder of Plugins ( Operating System plugins)
Abdullah&Hiatham 05/02/2011 12
Operating System plugins(Cont.)
Example of Fedora-14.appl plugin file:
name:
fedora-14 # Name of Plugin.
os:
name: fedora # Name of OS.
version: 14 # OS Version.
Fedora Operating System Plugin Usage:
In order to run the Fedora Plugin, the following command has to be used;
boxgrinder-build fedora-14.appl
Abdullah&Hiatham 05/02/2011 13
Platform plugins (Cont.)
Platform plugins convert intermediary deliverables produced by the operating system plugin into a selected platform. A platform could be VMware vSphere or Amazon EC2 for example.
There are three kinds of such plugin;
VMware plugin
VirtualBox Plugin
EC2 plugin
EC2 Platform Plugin:
This plugin creates a EC2 disk image. Note that created image isn’t a bundled AMI, it is a disk image prepared to be bundled and delivered by the S3 plugin.
Supported operating Systems:
Fedora – all versions
RHEL – all versions
CentOS – all versionsAbdullah&Hiatham 05/02/2011 14
Platform plugins (Cont.)
EC2 Platform Plugin (Continue):
EC2 Platform Plugin Configuration:
Configuration for this kind of plugins is not required
Ec2 Platform Plugin Usage:
boxgrinder-build applince-name.appl –p ec2
Abdullah&Hiatham 05/02/2011 15
Delivery plugins
Delivery plugins:
A delivery plugin moves the deliverables from a platfrorm or operating system plugin to a selected location type. This could be a local directory, SFTP server, Amazon CloudFront or an Amazon S3 bucket.
There are Four types of delivery Plugins;
Local plugin
SFTP plugin
S3 plugin
EBS Plugin
*The focus will be on the third type “S3 Plugin” as we had dealt with this term when we
used Amazon Web Services.
Abdullah&Hiatham 05/02/2011 16
Delivery plugins(Cont.)
S3 Delivery Plugin:
This plugin delivers artifacts to a S3 bucket. The plugin is able to deliver artifact in three types:
S3: a packaged (.tgz) image with metadata – good for distribution,
Cloudfront: a packaged image with metadata (same as for s3 type) for public download using CloudFront – great for distribution, you need to have CloudFront enabled for your account,
Ami: creates an AMI from selected image and registers it in Amazon EC2. After that the AMI will be visible for you as a private image and ready to run. This type is only available for images in EC2 format (converted using “-p ec2” switch).
Supoerted Operating Systems:
Fedora – all versions
RHEL – all versions
CentOS – all versions
S3 Delivery Plugin Supported Platforms:
EC2
Abdullah&Hiatham 05/02/2011 17
Delivery plugins(Cont.)
S3 Delivery Plugin Configuration:
This is how would the configuration section of this plugin look like:
In our Example:
plugins:
s3:
access_key: AWS_ACCESS_KEY # (required)
secret_access_key: AWS_SECRET_ACCESS_KEY # (required)
bucket: BoxGrinder # (required)
account_number: 6641-7087-8440 # (required)
path: /images # default: / The path were the image created is located.
cert_file: cert-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem
key_file: pk-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem
host: http://s3.amazonaws.com
Abdullah&Hiatham 05/02/2011 18
Delivery plugins(Cont.)
S3 Delivery Plugin (Continue):
S3 Delivery Plugin Example:
EC2 AMI for appliance-name.appl:
“boxgrinder-build jeos.appl -p ec2 -d ami” Command
Abdullah&Hiatham 05/02/2011 19
EBS Delivery plugins
This plugin delivers appliance as EBS-based AMI to AWS. Note that only appliances converted to EC2 format using EC2 platform plugin can be delivered as EBS AMI’s.
*This plugin can only be used on instances running on EC2. This plugin will not work on your local host because we need to mount EBS volume to copy the data and we cannot do a remote mount. You can use meta appliance AMI to create EBS AMI’s.
Supported Operating Systems:
Fedora – all versions
RHEL – Ver. 6
EBS Delivery Plugin Supported Platforms:
EC2
EBS Delivery Plugin Configuration:
This is how would the configuration section of this plugin look like:
Abdullah&Hiatham 05/02/2011 20
EBS Delivery plugins(Cont.)
plugins:
ebs:
access_key: AWS_ACCESS_KEY # required
secret_access_key: AWS_SECRET_ACCESS_KEY # required
account_number: 6641-7087-8440 # required
delete_on_termination: false # default: true
(Note that the delete_on_termination flag is used to specify if the root volume should be deleted after the instance is terminated)
EBS Delivery Plugin Example:
EBS-based AMI for jeos.appl:
“boxgrinder-build appliance-name.appl -p ec2 -d ebs” CommandAbdullah&Hiatham 05/02/2011 21
What is OpenStack?OpenStack is a collection of open source technologies delivering a massively
scalable cloud operating system. OpenStack is currently developing two interrelated projects: OpenStack Compute and OpenStack Object Storage.
OpenStack Compute is software to provision and manage large groups of virtual private servers, and OpenStack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.
Backed by Rackspace, NASA, Dell, Citrix, Cisco, Canonical and over 50 other organizations, OpenStack has grown to be a global software community of developers, technologists, researchers and corporations collaborating on a standard and massively scalable open source cloud operating system.
History:
In July 2010, Rackspace Hosting and NASA jointly launched a new open source cloud initiative known as OpenStack. The mission of the OpenStack project is to enable any organization to create and offer cloud computing services running on standard hardware. The community's first official release, code-named Austin, was made available just 4 months later with plans to release regular updates of the software every few months.
Abdullah&Hiatham 05/02/2011 22
Why OpenStack?
Control and Flexibility:
Open source platform means you’re never locked to a proprietary vendor, and modular design can integrate with legacy or third-party technologies to meet your business needs. Hypervisor support for Microsoft Hyper-V, Citrix XenServer, Xen, KVM, VMWware ESX, LXC, QEMU, and UML.
Industry Standard:
More than 60 leading companies from over a dozen countries are participating in OpenStack, including Cisco, Citrix, Dell, Intel and Microsoft, and new OpenStack clouds are coming online across the globe.
Proven Software:
Running the OpenStack cloud operating system means running the same software that today powers some of the largest public and private clouds in the world.
Compatible and Connected:
Compatibility with public OpenStack clouds means enterprises are prepared for the future—making it easy to migrate data and applications to public clouds when conditions are right—based on security policies, economics, and other key business criteria.
Abdullah&Hiatham 05/02/2011 23
Why OpenStack?(Cont.)
The OpenStackCloud:
OpenStack is a collaborative software project designed to create freely available code,badly needed standards, and common ground for the benefit of both cloud providers and cloud customers. OpenStack is currently three projects;
*OpenStack Compute
*OpenStack Object Storage
*OpenStack Image Service
OpenStack Compute:
OpenStack Compute is open source software designed to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. It gives you the software, control panels, and APIs required to orchestrate a cloud, including running instances, managing networks, and controlling access through users and projects.
Abdullah&Hiatham 05/02/2011 24
OpenStack Compute (Cont.):
Popular Use Cases:
* Service providers offering an IaaS compute platform
* IT departments provisioning compute resources to teams and projects
* Processing big data with tools like Hadoop
* Scaling compute up and down to meet demand for web resources and applications
Key Features:
*Complete multi-tenant cloud computing platform scaling to tens of thousands of compute nodes.
*Rich, well documented native OpenStack API with strong ecosystem and support for the Amazon EC2 API.
*Control panels that make it easy to move these workloads between data centers, and even different service providers.
Abdullah&Hiatham 05/02/2011 25
OpenStack Object Storage:
OpenStack Object Storage (code-named Swift) is open source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. It is not a file system or real-time data storage system, but rather a long-term storage system for a more permanent type of static data that can be retrieved, leveraged, and then updated if necessary.
Primary examples of data that best fit this type of storage model are virtual machine images, photo storage, email storage and backup archiving.
Popular Use Cases: * Service providers offering IaaS storage platform * Integrates with OpenStack Compute to store server images * Document storage * Back-end for Microsoft SharePoint * Archive platform for log files and other data with long retention periods * Store web images and thumbnails
Abdullah&Hiatham 05/02/2011 27
OpenStack Object Storage (Cont.)
Key Features:
* Complete multi-tenant cloud storage platform
* Massive scalability to multi-petabyte size and billions of objects
* Able to store objects of unlimited size
OpenStack Image Service:
OpenStack Image Service (code-named Glance) provides discovery, registration, and delivery services for virtual disk images. The Image Service API server provides a standard REST interface for querying information about virtual disk images stored in a variety of back-end stores, including OpenStack Object Storage. Clients can register new virtual disk images with the Image Service, query for information on publicly available disk images, and use the Image Service's client library for streaming virtual disk images.
Abdullah&Hiatham 05/02/2011 28
OpenStack Image Service (Cont.)
A multi-format image registry, OpenStack Image Service allows uploads of private andpublic images in a variety of formats, including:
* Raw
* Machine (kernel/ramdisk outside of image, a.k.a. AMI)
* VHD (Hyper-V)
* VDI (VirtualBox)
* qcow2 (Qemu/KVM)
* VMDK (VMWare)
* OVF (VMWare, others)
Popular Use Cases:
* Service providers offering an IaaS virtual machine image registry
* Enterprises creating a low cost DR/BCP platform to backup their virtual server images
* ISVs offering pre-configured optimized images of their softwareAbdullah&Hiatham 05/02/2011 29
OpenStack Image Service (Cont.)
Key Features:
*Configured to use standard command line and configuration option processing, making use of paste.deploy configuration
*Registry database is under version control, with migration files allowing upgrade and downgrade of the registry database.
*Versatile combination of "disk format" and "container format" fields, allowing more than just EC2-style image formats to be stored in the registry.
Abdullah&Hiatham 05/02/2011 30
References
• http://cs.uccs.edu/~cs526/cloudcomputing/intro.html
• http://fedoraproject.org/wiki/Features/BoxGrinder
• http://boxgrinder.org/
• http://www.openstack.org/index.php
Abdullah&Hiatham 05/02/2011 31