Who am I?
Infrastructure Engineer
Puppet User for 2+ years
Puppet Certified Professional
Linux user for 10+ years (Slackware!)
Metal Head \m/
The Marionette Collective (MCollective) is a framework to build server orchestration or parallel job execution
Works perfectly along side Puppet providing real-time command execution across your infrastructure
This is what it really is…
MCollective ArchitectureServers == Puppet Agent
Clients
Middleware == Puppet Master
Client Middleware
Server
Server
Server
Server
Servers
Systems that can be controlled via MCollective
Run the daemon mcollectived
puppet agent == mcollectived
Multiple Agent Plugins
small Ruby libraries that enable custom command execution
Clients
Clients send and receive requests to servers
Admin server for managing MCollective
Security and connector plugin
mco command-line client
interactive usage or in scripts
Middleware
!
Clients and servers communicate through the middleware system (ActiveMA, RabbitMQ)
MCollective uses a publish/subscribe system for all communication
Sends and receives messages using STOMP
Used for transporting messages between clients and servers
Message Flow
Broadcast paradigm
Messages are sent from the administrator account on the client server to the middleware server and broadcast to all nodes
Each agent is both a publisher and subscriber
Plugins
Enable MCollective to execute commands on the entire collective
Provide the functionality to use with MCollective
puppet agent, service, process, package, etc.
Integration with Puppet
Pre-installed and configured with Puppet Enterprise
Puppet Labs modules for opensource
https://github.com/puppetlabs/puppetlabs-mcollective
Orchestrating Puppet
MCollective can be used to control the Puppet Daemon
The performance of a Puppet Master is related to the max concurrent Puppet nodes that are applying a catalog
MCollective can determine how many servers are currently enabled and applying catalogs
mco puppet$ mco help puppet [14/180] !Schedule runs, enable, disable and interrogate the Puppet Agent !Usage: mco puppet [OPTIONS] [FILTERS] <ACTION> [CONCURRENCY|MESSAGE] Usage: mco puppet <count|enable|status|summary> Usage: mco puppet disable [message] Usage: mco puppet runonce [PUPPET OPTIONS] Usage: mco puppet resource type name property1=value property2=value Usage: mco puppet runall [--rerun SECONDS] [PUPPET OPTIONS] !The ACTION can be one of the following: ! count - return a total count of running, enabled, and disabled nodes enable - enable the Puppet Agent if it was previously disabled disable - disable the Puppet Agent preventing catalog from being applied resource - manage individual resources using the Puppet Type (RAL) system runall - invoke a puppet run on matching nodes, making sure to only run CONCURRENCY nodes at a time runonce - invoke a Puppet run on matching nodes status - shows a short summary about each Puppet Agent status summary - shows resource and run time summaries
Controlled Nodes
Query and run Puppet from any node that has the client installed:
$ mco puppet count Total Puppet nodes: 9 ! Nodes currently enabled: 9 Nodes currently disabled: 0 !Nodes currently doing puppet runs: 0 Nodes currently stopped: 9 ! Nodes with daemons started: 9 Nodes without daemons started: 0 Daemons started but idling: 9
Puppet Status
$ mco puppet status ! * [ ============================================================> ] 7 / 7 ! db02.example.com: Currently idling; last completed run 4 minutes 50 seconds ago fpvcs01.example.com: Currently idling; last completed run 24 minutes 44 seconds ago puppet.example.com: Currently idling; last completed run 6 minutes 48 seconds ago mco01.example.com: Currently idling; last completed run 24 minutes 35 seconds ago fp01.example.com: Currently idling; last completed run 4 minutes 55 seconds ago web01.example.net: Currently idling; last completed run 24 minutes 16 seconds ago mail02.example.com: Currently idling; last completed run 3 minutes 17 seconds ago !Summary of Applying: false = 7 Summary of Daemon Running: running = 7 Summary of Enabled: enabled = 7 Summary of Idling: true = 7 Summary of Status: idling = 7 !Finished processing 7 / 7 hosts in 96.09 ms
Running PuppetBasic Puppet run:
$ mco puppet runonce!
A noop run:$ mco puppet runonce —noop!
A run when noop is set:$ mco puppet runonce —no-noop!
A splay run:$ mco puppet runonce —splay —splaylimit 120
Filters
Host Filters -W, --with FILTER Combined classes and facts filter -S, --select FILTER Compound filter combining facts and classes -F, --wf, --with-fact fact=val Match hosts with a certain fact -C, --wc, --with-class CLASS Match hosts with a certain config management class -A, --wa, --with-agent AGENT Match hosts with a certain agent -I, --wi, --with-identity IDENT Match hosts with a certain configured identity
$ mco puppet runonce web01.example.com * [ ============================================================> ] 9 / 9
Invoking Actions with filters
Facts $ mco find -F osfamily=RedHat $ mco find —with-fact operatingsystemmajrelease=5
Classes
Puppet provides a list of classes applied to a node: $ mco find —with-class /apache/
Agent Runs $ mco puppet runall 1 2013-01-16 16:14:26: Running all nodes with a concurrency of 1 2013-01-16 16:14:26: Discovering enabled Puppet nodes to manage 2013-01-16 16:14:29: Found 2 enabled nodes 2013-01-16 16:14:32: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:37: dev1.example.net schedule status: Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false' command 2013-01-16 16:14:38: 1 out of 2 hosts left to run in this iteration 2013-01-16 16:14:40: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:44: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:48: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:52: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:14:56: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:00: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:04: Currently 1 node applying the catalog; waiting for less than 1 2013-01-16 16:15:08: Currently 1 node applying the catalog; waiting for less than 1
Misc Plugin Examples$ mco process list java -I dradis.example.com ! * [ ============================================================> ] 1 / 1 ! dradis.example.com ! PID USER VSZ COMMAND 1380 logstash 1.794 GB /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -X 1436 elasticsea 2.298 GB /usr/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=t !!Summary of The Process List: ! Matched hosts: 1 Matched Processes: 2 Resident Size: 146.453 MB Virtual Size: 4.092 GB !!Finished processing 1 / 1 hosts in 113.93 ms
$ mco package bash status -I db02.example.com ! * [ ============================================================> ] 1 / 1 ! db02.example.com: bash-4.1.2-15.el6_4.x86_64 !Summary of Arch: ! x86_64 = 1 !Summary of Ensure: ! 4.1.2-15.el6_4 = 1 !!Finished processing 1 / 1 hosts in 128.67 ms
InventoryRetrieve all facts, classes, and agent plugins:
!$mco inventory web01.example.com Agents: discovery filemgr package puppet rpcutil service thanks ! Data Plugins: agent fstat puppet resource service ! Configuration Management Classes: agent epel epel::params mcollective mcollective::common mcollective::common::config mcollective::common::config::connector::activemq mcollective::defaults mcollective::server mcollective::server::config mcollective::server::config::connector::activemq mcollective::server::config::factsource::yaml mcollective::server::config::rpcauditprovider::logfile mcollective::server::config::rpcauthprovider::action_policy mcollective::server::config::securityprovider::ssl mcollective::server::install mcollective::server::service ntp ….. ! Facts: architecture => x86_64 augeasversion => 1.0.0 bios_release_date => 04/17/2013 bios_vendor => Xen bios_version => 4.0.1 blockdevice_xvda_size => 21559771136 blockdevice_xvdb_size => 17179869184 blockdevices => xvda,xvdb
inventory do format "%40s:\t\t%20s\t\t%s" ! fields { [ identity, facts["operatingsystemrelease"], facts["ipaddress"] ] } end
$ mco inventory -C /epel/ —script inventory.rb mco01.example.com: 6.5 192.168.2.25 web02.example.com: 6.1 192.168.2.30 mon01.example.com: 6.5 192.168.2.28 db01.example.com: 6.5 192.168.2.40 db02.example.com: 6.3 192.168.2.60 mail02.example.com: 6.5 192.168.2.50
Inventory Reports
Top Related