Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLON
-
Upload
outlyer -
Category
Technology
-
view
1.886 -
download
0
Transcript of Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLON
Push Infrastructure with Ansible Paul Angus
Cloud Architect & Chief Technology Strategist ShapeBlue [email protected]
Twitter: @CloudyAngus
@CloudyAngus
v How we use Ansible to push out entire cloud infrastructures
v A bit of background v How we use Ansible
v Work flow v Code snippets
Push Infrastructure with Ansible
@CloudyAngus
v Cloud Architect & Chief Technology Strategist for ShapeBlue
v Apache CloudStack Committer v Specialise in….
v Designing and deploying enterprise and public clouds v Helping organisations use their cloud
v Involved with CloudStack before donation to Apache Foundation
v Designed clouds for Orange, TomTom, PaddyPower, Ascenty, BSkyB
About Me
Unofficial Ansible Evangelist/Cheerleader
@CloudyAngus
“ShapeBlue are expert builders of public & private clouds. They are the leading global
Apache CloudStack integrator & consultancy”
About ShapeBlue
@CloudyAngus
Push Infrastructure with Ansible
What is CloudStack?
@CloudyAngus
CloudStack is a open source IaaS platform. Hypervisor agnostic -‐ KVM, vSphere, XenServer, LXC, OVM, Baremetal
CloudStack orchestrates hypervisors and network appliances to give simple control to complex tasks through API or web GUI.
(Yes, it’s like OpenStack)
What is CloudStack
@CloudyAngus
Think ‘your own Amazon Web Services’
Public clouds (SPs/MSPs) General public can create or log into instances themselves
Private Clouds (Enterprises) Anyone who wants to be able to orchestrate their environment
Hybrid Clouds (Enterprises) Balance/share load between their own DC and a Public Cloud
Who uses CloudStack
@CloudyAngus
This is a UI (for Linux guys
who’ve forgotten).
What is CloudStack
@CloudyAngus
It’s OK it has an API cloudmonkey create network name=DemoNet displaytext=DemoNet zoneid=$ZONEID networkofferingid=$NEWORKOFFERINGID account=$ACCOUNTNAME domainid=$DOMAINID
cloudmonkey list networks domainid="$DOMAINID" filter=id,name | jq '.network[] | select(.name=="DemoNet") | .id‘
cloudmonkey deploy virtualmachine networkids="$NETWORKID" serviceofferingid="$SERVICEOFFERINGID" templateid="$TEMPLATEID" zoneid="$ZONEID" account="$ACCOUNTNAME" domainid="$DOMAINID" startvm=true displayname=HTTPServer1 displayvm=true name=HTTPServer1
What is CloudStack
@CloudyAngus
Push Infrastructure with Ansible
Why? (Duuurrr)
@CloudyAngus
CEO
Why
“Building CloudStack environments using Ansible?
Are you just having fun with Ansible?”
@CloudyAngus
Why
Talented Cloud Architect
“Noooo, if we can automate the building of environments using a powerful, simple and agentless technology we can make building at scale easy
while ensuring that our results are consistent and repeatable.”
“Building CloudStack environments using Ansible? Are you just having fun with
Ansible?”
@CloudyAngus
Why
CEO “That would be excellent. Go ahead. Oh, and here’s a pay rise”
“Noooo, if we can automate the building of environments using a powerful, simple and agentless technology we can make building at scale easy
while ensuring that our results are consistent and repeatable.”
@CloudyAngus
Some of that might actually have happened.
Disclaimer
Why
@CloudyAngus
CSForge™ v CSForge delivers the rapid deployment of a standardised CloudStack powered IaaS
cloud for small production deployments, or medium scale POCs or pilots. The framework can be used as a basis for public cloud or enterprise private cloud deployments
Production v Cloud-‐scale environments initial deployment typically 24 – 100s of hosts v Often multi-‐hypervisor
Why
@CloudyAngus
Test/Dev
Need to be able to create full environments to test: v CloudStack release candidates v CloudStack features v ShapeBlue patches
Why
@CloudyAngus
Why Ansible v Technical:
v Client/Server architecture not required v Only SSH connectivity required (password or public/private keys) v …making it easier to build virgin environments v Modules can be in any language capable of returning JSON or key=value text pairs v Has an API
v User:
v Much shallower learning curve v Don’t need to learn a programming language (i.e. Ruby)
v Not as many pre-‐existing playbooks (recipes/manifests) about, but improving with Ansible Galaxy
@CloudyAngus
Typical Logical Production Topology
Management (1Gb)
CIMC (1GB)
CIMC (1GB)
Management (1GB LACP Bond)
WWW
Compute Hosts
Storage Nodes
GuestPublicStorage
(10GB LACP Bond)
Management Hosts
Management1GB Active/Passive Bond
Load Balancers
DNS/NTP Servers
MySQL Master & Slave
ACS Management Servers
DeploymentServer
Management(1GB Active/Passive Bond)
Storage
(10GB LACP Bond)
iDRAC (1GB)
Storage (10GB LACP Bond)
Public
Cloud Compute Hosts
Storage Link to ACS Managers
@CloudyAngus
v 3 zones v 2 geographic
locations v Upgrade done,
then tests run for a week. Then VRs restarted
Client Test Environment
CCP3.0.7B
MySQL
CPBM 2.2
MySQL
ESXi 1a
ESXi 1b
ESXi 1cvCenter Appliance
ESXi 2a
ESXi 2b
ESXi 2cvCenter ApplianceZone 1 (local)
Zone 2 (local)
Zone 3 (remote)
NFS
NFS
VPN VPN
ESXi 2a
ESXi 2b
ESXi 2cvCenter Appliance
NFS
@CloudyAngus
Using Ansible to Deploy CloudStack Environments
How? (variables, I Love ‘em)
@CloudyAngus
# Copyright (C) ShapeBlue Ltd -‐ All Rights Reserved # Unauthorized copying of this file, via any medium is strictly prohibited # Proprietary and confidential # Released by ShapeBlue <[email protected]>, April 2014 -‐-‐-‐ -‐ name: Ensure selinux python bindings are installed yum: name=libselinux-‐python state=present -‐ name: Ensure the Apache Cloudstack Repo file is configured template: src=cloudstack.repo.j2 dest=/etc/yum.repos.d/cloudstack.repo -‐ name: Ensure selinux is to permissive command: setenforce permissive changed_when: false -‐ name: Ensure selinux is set permanently selinux: policy=targeted state=permissive -‐ name: Ensure CloudStack packages are installed yum: name=cloudstack-‐management state=present
-‐ name: Ensure MySQL Client is present yum: name=mysql state=present -‐ name: Ensure vhd-‐util is present get_url: url="{{ vhdutil_url }}" dest=/usr/share/cloudstack-‐common/scripts/vm/hypervisor/xenserver/vhd-‐util mode=0755 -‐ name: Ensure CloudStack Usage Service is installed yum: name=cloudstack-‐usage state=present -‐ name: Ensure CloudStack Usage Service is started service: name=cloudstack-‐usage state=started -‐ include: ./../galera-‐cluster/tasks/main.yml when: "{{ db_type }} == 'galera'" -‐ include: ./setupdb.yml
CloudStack Management Server Role
@CloudyAngus
# Copyright (C) ShapeBlue Ltd -‐ All Rights Reserved # Unauthorized copying of this file, via any medium is strictly prohibited # Proprietary and confidential # Released by ShapeBlue <[email protected]>, April 2014 [cloudstack] name=cloudstack baseurl=http://{{ acs_build_repo }}/{{ acs_build_path }} enabled=1 gpgcheck=1
cloudstack.repo.j2
@CloudyAngus
# ansible-‐playbook management_vms_build.yml -‐i acs_demo_hosts -‐-‐extra-‐vars "acs_build_path=tescng/centos/master"
Playbook Arguments
@CloudyAngus
Create bare VM +
yum install git
git clone mega repo with roles
etc.
Install & configure Ansible (included in repo)
Update hosts and group_vars
Create Deployment
server (from role) locally
PXE boot hosts/mgmt VMs to
bare OS
Push application configuration to VMs and Hosts
Building Environments
@CloudyAngus
Distributed Architecture
TEST STAGE PROD
Deployment Srvrwith cloned repo
Deployment Srvrwith cloned repo
Deployment Srvrwith cloned repo
@CloudyAngus
v PXE VM against Ansible created KickStart file
v DNS Servers (Bind 9) v Galera Cluster (Maria DB10 – any size) v MySQL Master/Slave v Clustered CloudStack Management Servers v HA Proxy (HA pair with keepalived)
Management Infrastructure Roles
@CloudyAngus
v PXE host against Ansible created KickStart or Answer file v KVM
v Straight configuration of networking and packages
v XenServer v XE commands via shell v Use ‘facts’ to determine installed patches
v vSphere v 99% configured through kickstart file v SSH enabled (through KickStart)
Hypervisor Host Roles
@CloudyAngus
Using Ansible to Deploy CloudStack Environments
Snippets
@CloudyAngus
[management_hosts:children] cloudstack_manager_hosts loadbalancer_hosts deployment_hosts db_hosts [kvm_hosts:children] pod1_cluster1 pod1_cluster2 [pod1_cluster1] 10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13 mgmt_netmask=255.255.255.0
Host File
@CloudyAngus
Host File [mysql_master_host]
10.101.1.6 pxemac=00:50:56:8E:19:20 hostname=mysqlmasterhost mgmt_ip=10.101.1.6 mgmt_netmask=255.255.255.0 serverid=1
[pod1_cluster1]
10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13 10.101.1.14 pxemac=00:1c:c0:ec:b5:43 hostname=kvm-‐p1-‐c1-‐2 mgmt_ip=10.101.1.14
Creating PXE files
@CloudyAngus
File format: AA:BB:CC:DD:EE:FF must be 01-‐aa-‐bb-‐cc-‐dd-‐ee-‐ff
-‐ name: create centos pxeconfig files
template: src=pxelinuxcfg-‐centos6.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower | replace(":","-‐") }} with_items:
-‐ "{{ groups.management_hosts }}“
-‐ name: create KVM pxeconfig files
template: src=pxelinuxcfg-‐kvm.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower | replace(":","-‐") }} with_items:
-‐ "{{ groups.kvm_hosts }}"
Creating PXE files
@CloudyAngus
# Ansible built pxeconfig script DEFAULT menu.c32 menu ctle ShapeBlue CSForge Deployment cmeout 10 LABEL {{ centos_iso_version }}-‐KVM kernel htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}/isolinux/vmlinuz append initrd=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}/isolinux/initrd.img ramdisk_size=15491 ip=dhcp ksdevice={{ ks_device }} ks=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ pxe_files_dir }}/{{ kvm_ks_file }} MENU LABEL {{ centos_iso_version }} for KVM
pxelinuxcfg-‐kvm.j2
@CloudyAngus
# Specifies the keyboard layout keyboard {{ keyboard_lang }} # Used with an HTTP install to specify where the install files are located url -‐-‐url htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }} # Assign a stacc IP address upon first boot & set the hostname network -‐-‐onboot yes -‐-‐device {{ ks_device }} -‐-‐bootproto dhcp -‐-‐noipv6 # Set the root password rootpw {{ mgmt_root_password }} mkdir /root/.ssh curl htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ publickey_file_name }} >> /root/.ssh/authorized_keys
kickstart.j2
@CloudyAngus
opcon subnet-‐mask {{ dhcp_netmask }}; opcon routers {{ dhcp_router }}; opcon broadcast-‐address {{ dhcp_broadcast }};
opcon domain-‐name-‐servers {{ int_dns_servers | join(", ") }}; opcon domain-‐name "{{ dns_domain }}"; subnet {{ dhcp_subnet }} netmask {{ dhcp_netmask }} { range {{ dhcp_range_starcp }} {{ dhcp_range_endip }}; # KVM Hosts
{% for host in groups['kvm_hosts'] %} host {{ hostvars[host]['hostname'] }} { hardware ethernet {{ hostvars[host]['pxemac'] | lower }}; fixed-‐address {{ hostvars[host]['mgmt_ip'] }}; } {% endfor %}
Creating dhcpd.conf
@CloudyAngus
v Reverse Zone – Format: 2 IN PTR blah1.domain.com.
; Address Records {% for host in groups['management_hosts'] %}
{{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}. {% endfor %} {% for host in groups['xenserver_hosts'] %} {{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}. {% endfor %}
DNS Entries into Zone File – Jinja2
@CloudyAngus
v Custom Filter to Split string import re
def split_string(string, seperator=' '): return string.split(seperator)
def split_regex(string, seperator_patern): return re.split(seperator_patern, string) class FilterModule(object):
''' A filter to split a string into a list. ''' def filters(self): return { 'split' : split_string,
'split_regex' : split_regex, }
Custom Filter
@CloudyAngus
v ansible.cfg
# set plugin path directories here, separate with colons
accon_plugins = /usr/share/ansible_plugins/accon_plugins
callback_plugins = /usr/share/ansible_plugins/callback_plugins
conneccon_plugins = /usr/share/ansible_plugins/conneccon_plugins
lookup_plugins = /usr/share/ansible_plugins/lookup_plugins
vars_plugins = /usr/share/ansible_plugins/vars_plugins
filter_plugins = /usr/share/ansible_plugins/filter_plugins:/CSForge/custom_plugins/filter_plugins
Add path if required
@CloudyAngus
v ACS_env_group_vars.yml
db_type: galera # opcons: mysql, mysql-‐masterslave, galera
v build_db_servers.yml -‐-‐-‐
-‐ hosts: db_hosts
roles:
-‐ { role: galera-‐cluster, when: "'{{ db_type }}' == 'galera'" }
-‐ { role: mysql, when: "'{{ db_type }}' != 'galera'" }
Conditional Roles
Using Ansible to Deploy CloudStack Environments
Paul Angus Cloud Architect ShapeBlue [email protected]
Twitter: @CloudyAngus