CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04...

21
#OpenCORD CORD How to build a POD and automate deployments

Transcript of CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04...

Page 1: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD

CORD How to build a POD and automate deployments

Page 2: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 2

Full POD: definition The minimum amount of hardware that can be used to perform a full test of the current CORD features

Fabric 4x white-box switches

Compute 3x standard x86 servers

Fabric switch

Fabric switch

Fabric switch

x86 server

x86 server

Mgmt L2 switch

x86 server

Fabric switch

Tricks ●  Avoid company proxy servers ●  Use suggested hardware

Suggested BOM includes:

●  Server QuantaGrid D51B-1U (2x Intel

E5-2630 v4 10C 2.2GHz 85W, 64GB of RAM 2133MHz DDR4, 2x hdd500GB)

●  40G NIC: Intel Ethernet Converged Network Adapters XL710 10/40 GbE PCIe 3.0, x8 Dual port

●  Switches: Accton 6712 - 32x40GE

Page 3: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 3

Software requirements

Dev node (operator laptops or 4th server) •  Ubuntu 16.04 (suggested) or Ubuntu 14.04 •  Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!)

Head node •  Ubuntu 14.04 •  Accessible from the dev node without password, sudoer - no password Compute nodes, switches •  We’ll take care of them!

Page 4: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 4

Network connectivity: user / data plane

Head node 1

Compute node 1

Compute node 2

Leaf 1 Leaf 2

Spine 1 Spine 2 Fabric 4x whitebox switches

Compute 3x standard x86 servers

Access devices Metro network

Page 5: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 5

Network connectivity: a complete view

Compute node 1

Head node

Spine 1

Spine 2 Internet

Linux mgmt to external

Fabric to leafs

Fabric to leafs

Linux mgmt to internal

Linux mgmt to internal

Compute node 2

Fabric to leafs

Linux mgmt to internal

Leaf 1

Leaf 2

Fabric

Fabric

Fabric

Fabric

Exte

rnal

net

wor

k L2

sw

itch

Mgmt

Mgmt

Mgmt

Mgmt

Operator / Dev machine

IPMI

IPMI

Inte

rnal

mgm

t L2

switc

h

Page 6: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 6

MAAS

Machine to be installed

An “evolution” of the old PXE servers (Basically DHCP server + TFTP)

Used to install an OS on remote

machines and to customize it

The OS is installed using the Ethernet mgmt interface, and it’s

started/stopped using the IPMI interface

PXE Client The node boots from the network

It gets an IP address from the DHCP server

MAAS installs the OS

Ethernet mgmt

IPMI

Page 7: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 7

Deployment steps

Head node

Compute node 2

Compute node 1

Leaf 1

Leaf 2

Spine 1

Spine 2

Operator / Dev machine

Internet

Steps to deploy:

1.  Download CORD repo on the dev machine

2.  Create the CORD dev VM on the dev machine

3.  Fetch CORD packages on the dev machine

4.  Push the software to the head node 5.  Deploy and configure the head node

6.  Reboot (to deploy) the compute

nodes and the switches

7.  Add your configurations Runs

●  OpenStack head node ●  ONOS ●  XOS ●  ... ●  MAAS

Page 8: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 8

Deployment steps

Head node

Compute node 2

Compute node 1

Leaf 1

Leaf 2

Spine 1

Spine 2

Operator / Dev machine

Internet

Steps to deploy:

1.  Download CORD repo on the dev machine

2.  Create the CORD dev VM on the dev machine

3.  Fetch CORD packages on the dev machine

4.  Push the software to the head node 5.  Deploy and configure the head node

6.  Reboot (to deploy) the compute

nodes and the switches

7.  Add your configurations Runs

●  OpenStack head node ●  ONOS ●  XOS ●  ... ●  MAAS

Page 9: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 9

Broadcast domains

Head node

Compute node 2

Compute node 1

Leaf 1

Leaf 2

Spine 1

Spine 2

Operator / Dev machine

Internet

Broadcast domain 1: external network -  Routable IP addresses -  Access to Internet -  Addresses given by the administrator

Broadcast domain 2: internal/management network -  NON routable IP addresses (only internal to the POD) -  NO access to Internet -  Addresses given to clients by DHCP of CORD MAAS

(configurable)

Page 10: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 10

Issues

Building a CORD POD requires ~3-4 hours (human interaction)

Is there a way to automatically reset a POD?

How do I do “CI”? What if I want to test a fresh installation with the latest changes every day?

Page 11: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 11

Automation: Jenkins integration

Head node

Compute node 2

Compute node 1

Leaf 1

Leaf 2

Spine 1

Spine 2

Operator / Dev machine

Internet

External network L2 switch

Page 12: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 12

Broadcast domains

Head node

Compute node 2

Compute node 1

Leaf 1

Leaf 2

Spine 1

Spine 2

Operator / Dev machine

Internet

External network L2 switch

DHCP server

Clients

Client

Page 13: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 13

Network connectivity: a complete view

Compute node 1

Head node

Spine 1

Spine 2 Internet

Linux mgmt to external

IPMI

Fabric to leafs

IPMI

Fabric to leafs

Linux mgmt to internal

Linux mgmt to internal

Compute node 2 IPMI

Fabric to leafs

Linux mgmt to internal

Leaf 1

Leaf 2

Fabric

Fabric

Fabric

Fabric

Inte

rnal

mgm

t L2

switc

h

Exte

rnal

net

wor

k L2

sw

itch

Mgmt

Mgmt

Mgmt

Mgmt

Operator / Dev machine

Page 14: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 14

ONF “Jenkins as a Service”

Partner CORD POD

IP-SEC VPN IP-SEC VPN

Page 15: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 15

Automated CORD POD

Fabric switch

Fabric switch

Fabric switch

x86 server

x86 server

Mgmt L2 switch

x86 server

Fabric switch

Dev + MAAS

IP-SEC VPN X Internet

IP-SEC VPN

What Jenkins does

1.  Re-installs Ubuntu on the head node through the additional MAAS APIs

2.  Downloads the latest source code 3.  Downloads a pre-defined POD

configuration

4.  Builds software and push it to the head node

5.  Reboots the compute nodes and the

switches

Page 16: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD

Jenkins scripts overview

Page 17: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 17

Automation building blocks

Jenkinsfile •  REPO_ROOT/build/Jenkinsfile.newBuildSystem CORD POD configurations •  https://gerrit.opencord.org/pod-configs

Jenkins deployment configurations •  pod-configs repository – deployment folder

Page 18: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD

Let’s have a look at them…

Page 19: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 19

Summary Deploy CORD is not that difficult… •  Not lots of software required •  Two commands, few reboots

Automation •  People love it! Why do work, if software does it for you? •  For CI: deploy, configure, test, reset •  For CD: develop your code and test it on the fly

Deployment on demand •  Offered to all ONF collaborators and partners •  We take care of the setup for you •  You just need a VPN connection to us

Page 20: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD 20

References Documentation

http://guide.opencord.org (!!!)

How to get help / works also from China

Mailing-list: [email protected]

Slack: slack.opencord.org

Presenter

Luca Prete / [email protected]

Page 21: CORD€¢ Ansible, Libvirt, Vagrant, Vagrant plugins (versions matter!) Head node • Ubuntu 14.04 • Accessible from the dev node without password, sudoer - no password Compute nodes,

#OpenCORD

Thank you!


Development with Vagrant

Development with Vagrant

vagrant - WordPress.tv · config.vm.box = "hashicorp/precise64" preconfigured machines, different flavors all have the same user: vagrant (password: vagrant) Base Boxes

vagrant - WordPress.tv · config.vm.box = "hashicorp/precise64" preconfigured machines, different flavors all have the same user: vagrant (password: vagrant) Base Boxes

DevOps with Vagrant and KVM/qemu Hiroshi Miura (@miurahr ... · Vagrant-KVM vs. -libvirt Features Vagrant-KVM Vagrant-libvirt KVM Yes Yes Xen N.A. Plan Remote N.A. Experimental ...

DevOps with Vagrant and KVM/qemu Hiroshi Miura (@miurahr ... · Vagrant-KVM vs. -libvirt Features Vagrant-KVM Vagrant-libvirt KVM Yes Yes Xen N.A. Plan Remote N.A. Experimental ...

Vagrant presentation

Vagrant presentation

Monitor - percona.com $ ssh vagrant@127.0.0.1 -p 2222 -o PreferredAuthentications=password Windows: 12

Monitor - percona.com $ ssh [email protected] -p 2222 -o PreferredAuthentications=password Windows: 12

3-node cluster using Vagrant & VirtualBox Ganeti Setup ... · vagrant up node3 vagrant ssh node1 gnt-node add -s 33.33.34.13 node3 The Vagrantfile is setup to where you can deploy

3-node cluster using Vagrant & VirtualBox Ganeti Setup ... · vagrant up node3 vagrant ssh node1 gnt-node add -s 33.33.34.13 node3 The Vagrantfile is setup to where you can deploy

SPDK Vagrant Box Install Instructions · vagrant@localhost:/vagrant# cd fio && git checkout fio-3.3 vagrant@localhost:/vagrant# ./configure vagrant@localhost:/vagrant# make vagrant@localhost:/vagrant#

SPDK Vagrant Box Install Instructions · vagrant@localhost:/vagrant# cd fio && git checkout fio-3.3 vagrant@localhost:/vagrant# ./configure vagrant@localhost:/vagrant# make vagrant@localhost:/vagrant#

Vagrant Intro

Vagrant Intro

Vagrant - "Pilot"

Vagrant - "Pilot"

Tech Talk - Vagrant

Tech Talk - Vagrant

Intro to vagrant

Intro to vagrant

Vagrant and ansible

Vagrant and ansible

Vagrant Plugin development

Vagrant Plugin development

VAGRANT - September 2012

VAGRANT - September 2012

The Vagrant Story

The Vagrant Story

Vagrant development environment

Vagrant development environment

Vagrant workshop 2015

Vagrant workshop 2015

Vagrant and-ansible

Vagrant and-ansible

Vagrant - Rohit Gupta · Tao of Vagrant Developers can check out any code repository and vagrant up. Systems Ops engineers can vagrant up and have a complete sandbox matching production.

Vagrant - Rohit Gupta · Tao of Vagrant Developers can check out any code repository and vagrant up. Systems Ops engineers can vagrant up and have a complete sandbox matching production.

AVAMAR UTILITY NODE REPLACEMENT PROCEDURE · User=admin 2. Log into the replacement utility node as user admin. 3. When prompted for a password, enter the admin password and press

AVAMAR UTILITY NODE REPLACEMENT PROCEDURE · User=admin 2. Log into the replacement utility node as user admin. 3. When prompted for a password, enter the admin password and press