Open Source Tool Chains for Cloud Computing
-
Upload
mark-hinkle -
Category
Technology
-
view
8.148 -
download
0
description
Transcript of Open Source Tool Chains for Cloud Computing
Open Source Toolchains for Managing Clouds
Monday, August 9, 2010
Mark R. HinkleVP of Community
Zenoss Inc. [email protected]@gmail.comTwitter: @mrhinkle
John M. WillisVP of ServicesOpscode Inc.
[email protected]@gmail.comTwitter: @botchagalupe
ABSENT
Monday, August 9, 2010
%whoami•Former Linux Desktop Advocate
•Former LinuxWorld Magazine Editor-in-Chief
•Open Management Consortium Conspirator
•Open Desktop Consortium Instigator
•Author - “Windows to Linux Business Desktop Migration” - Thomson
•NetDirector Project - Open Source Configuration Management Project
Monday, August 9, 2010
Today’s
Agenda
Definitions (Toolchains, Systems Management, Cloud Computing)
Bad Jokes
Overview of Open Source Management Tools
Culture Changes
Alien Autopsy Photos
Example Cloud Computing Toolchains
Monday, August 9, 2010
ToolchainA set of programs where the output of one program forms the input of another program.
Monday, August 9, 2010
Open Source Management Tools Adoption
• 98% of enterprises use open source systems management tools
• 76% indicate they prefer to use open source whenever possible
• Compelling factors for using open source is flexibility followed by cost savings
• 50% are already using some form of cloud technology including but not limited to hosted applica@ons, Amazon Web services and/or hosted storage
• Top IT management priori@es for 2010: monitoring, configura4on management, patching and
Source: 2010 ZenossOpen Source Management Survey
Monday, August 9, 2010
Cloud ComputingHype is Becoming a
Reality•UBS says Web Services will be a $15 billion
+ market by 2014
• IBM says Cloud Computing will be a $126 billion by 2012
• 20% of Businesses won’t have IT Assets by 2012 - Gartner
Bottom line a large portion of our infrastructure no longer lives in our data center... but
we gotta manage it.
Monday, August 9, 2010
Systems Management Disciplines
Sets the parameters for servers, starts and stops services, rotates logs and other menial task
Monitoring queries the servers for overall health and alerts administrators to problems
Configuration Management and Automation
Installation of operating systems and other software
Provisioning
Monitoring
Monday, August 9, 2010
How to Choose Open Source Cloud Tools
• Open Source (OSI Approved License)
• Manage Legacy Infrastructure and Cloud
• Extensible (Plugins, accept code contributions)
• Vibrant Communities (activity in Downloads, Forums, Extensions)
• Client/server (or at least network aware)
Monday, August 9, 2010
Comparison of Provisioning Tools
Year Started Language License Installation
Targets
Cobbler 2007
Fully Automatic Installation (FAI) 2000
Kickstart ?
OpenQRM 2005
Spacewalk 2008
Viper 2008
Python GPL Red Hat, Fedora, OpenSuSE, Debian, Ubuntu
Perl GPL Debian
Python GPL Most .deb and Fedora based Linux
PHP GPL Linux, Solaris, Windows
Perl, Python, Java GPL Fedora, CentOS
Perl GPL Debian
Monday, August 9, 2010
Comparison of Configuration Management & Automation Tools
Year Started Language License Client/Server Backing
AutomateIT 2009
bcfg2 2003
Cfengine 1993
chef 2009
Puppet 2004
Ruby GPL No None
BSD BSD Yes Argonne National Labs?
Apache Apache Yes Cfengine Inc.
Ruby Apache Chef Solo - NoChef Server - Yes Opscode
Ruby GPL Yes PuppetLabs
Monday, August 9, 2010
Comparisons of Open Source Monitoring Tools
Year Started License Language Monitoring
Type Collection Methods
Cacti
Nagios
OpenNMS
Zabbix
2001 GPL PHP Performance SNMP, Syslog
1999 GPL C/PHP Availability SNMP, TCP, ICMP, IPMI Syslog
2000 GPL Java Availability, Performance, SNMP, JMX, HTTP
2001 GPL C/PHPAvailability,
Performance, and more
SNMP, TCP, ICMP, IPMI, Synthetic Transactions
Zenoss 2005 GPL PythonAvailability,
Performance, Event Management
SNMP, SSH, Syslog, Event Management, Synthetic
Transactions
Monday, August 9, 2010
CloudOps: Change in Culture
GapingVoid - http://www.gapingvoidgallery.com/product_info.php?products_id=1643&osCsid=bc3tdqg6fuh8gato04m9obr0o1
Monday, August 9, 2010
Old Systems Management
Practices•Human Powered, Labor Intensive
•Repetitive Tasks
•“Meat Cloud”
•How many servers do you manage per admin?
Monday, August 9, 2010
Cloud Changes Everything • Hyperscalable
• Hardware Abstraction
• Dynamic Infrastructure
• Geography Independent
• Fast & Flexible
• How many cloud instances do you manage per admin?
Monday, August 9, 2010
The Myth of the NinesAvailability % Downtime per Year Downtime per Month Downtime per Week
99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes
99.95% 4.38 hours 21.56 minutes 5.04 minutes
99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes
99.999% (five nines) 5.26 minutes 25.9 seconds 6.05 minutes
99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds
• Average polling interval for monitoring? 5 minutes? • Even super human operations people can’t be alerted and take action in under 5 minutes. • One outage per year could drop service level to three nines or worse.
Monday, August 9, 2010
Copyright © 2010 Opscode, Inc - All Rights Reserved
P
Automate or get run over.
17Monday, August 9, 2010
Cultural ChangesAgile IT and DevOps movements
• Operations and Developers should collaborate with each other to deliver excellent products
• Systems Administrators need to be come Systems Engineers building automated, fault tolerant systems not just maintaining infrastructure
• More frequent changes, more outages (albeit short) to rapidly improve IT products and services
• Process, version control, and automation are important
Monday, August 9, 2010
AutomateITChef
Control TierPuppet
KickstartSpacewalk
CobblerOpenQRM
Configuration Management and
Automation
GangliaNagios
OpenNMSZabbix
Zenoss Core
MonitoringProvisioning
Systems Management Tools for Cloud Computing
Monday, August 9, 2010
“Off the shelf” Open Source Toolchains
• OpenQRM and Nagios
• Cobbler and Puppet
• Zenoss & [Chef, Cfengine and Puppet]
• Spacewalk and Cobbler
• OpenNMS and Rancid
• OpenNMS and Puppet
Monday, August 9, 2010
DevOps ToolChain Project
Project centered around how to automate and improve infrastructure management using Agile/DevOps methodologies
Discussion Topics • Open questions on unified pipe architecture
• Distribution methods: package vs file, rscyn/murder vs yum/rpm vs DFS
• Configuration management: RPMs vs puppet/cfengine/chef tool?
• Rollback methodologies for package and config management tools
• Controlling and timing package release and config management tools
• Log management (aggregating, crunching, charting)Change detection
http://code.google.com/p/devops-toolchain/
Monday, August 9, 2010
Cloud Computing
Changes Everything
•MeatCloud, Can’t Keep up with Cloud Computing
•Devops & Agile IT Philosophy
•Script Repetitive Tasks
•Automate, Automate, Automate
Monday, August 9, 2010
Example Cloud ToolChain•Multiple Cloud Providers
•Mix and Match and match tools
•Portability, Flexibility, and
Monday, August 9, 2010
Example - Geeknet• Servers are automatically built using configuration
management software
• Discovery tool finds infrastructure and populates a CMDB then spits out information to scripts that translate information to BIND configurations for DNS
• Monitoring tool adds hosts to polling tool to check servers for availability
• As infrastructure changes systems are updated automatically
• Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction
Hundreds of servers, serving web, databases, and other infrastructure for some of the world’s most highly trafficked websites – over 40 million visitors per month.
Monday, August 9, 2010
Summary
• Automate to improve service, apply leverage
• Rethink how your operations works
• Choose tools that can extend and adapt to new types of infrastructure (what does cloud look like in 2015?)
Monday, August 9, 2010
Questions? Comments?
Monday, August 9, 2010