Monitoring with

First of all...

First of all...

Why we (me) don’t like Nagios?

• No active-active cluster

Why we (me) don’t like Nagios?

• No active-active cluster (nor easy active-passive one)

Why we (me) don’t like Nagios?

• No active-active cluster (nor easy active-passive one)

Why we (me) don’t like Nagios?

• No active-active cluster (nor easy active-passive one)

• Restart after adding a host (auto-scaling anyone??)

Why we (me) don’t like Nagios?

• No active-active cluster (nor easy active-passive one)

• Restart after adding a host (auto-scaling anyone??)

• (Really) difficult to hack on

Why we (me) don’t like Nagios?

• The master executes the checks (all of them!)

Why we (me) don’t like Nagios?

• The master executes the checks (all of them!)

• Painful config reloads if using slaves

Why we (me) don’t like Nagios?

• The master executes the checks (all of them!)

• Painful config reloads if using slaves

• (Quite) hard to configure

Why we (me) don’t like Nagios?

• First release: March 14, 1999

Why we (me) don’t like Nagios?

• First release: March 14, 1999

What’s Sensu?

• A monitoring framework

• They say “monitoring router”

• It takes results of checks and passes them to handlers

Sensu facts

• Active-active cluster

Sensu facts

• Active-active cluster

• Hosts automagically registered when agent starts

Sensu facts

• Active-active cluster

• Hosts automagically registered when agent starts

• Written in Ruby

Sensu facts

• Clients execute all the checks (all of them!)

Sensu facts

• Clients execute all the checks (all of them!)

• No need to have slaves (but you can if you want to)

Sensu facts

• Clients execute all the checks (all of them!)

• No need to have slaves (but you can if you want to)

• Configured with small JSON files

Sensu facts

• Clients execute all the checks (all of them!)

• No need to have slaves (but you can if you want to)

• Configured with small JSON files

• Compatible with Nagios plugins!

Sensu facts

• IT’S 2014

Sensu facts

• IT’S 2014

What does Sensu provide?

• Agent (client) to gather check results

• Server to take actions (or not) on these results

• REST API to query the state

• Optional dashboard to query the API

What does Sensu NOT provide?

• Storage for the results history

• Notifications

• Graphing

Architecture overview

Two modes of operation

Subscription

subscription messages

Subscription

check request

Subscription

check results

Standalone

Standalone

check results

• Modes can be mixed

• The client also listens for results in a socket– Throw JSON to it from your application

Handlers

• Each check can have its own handler

• Sensu considers only results flagged as “metrics” and not-ok’s

Handler types

• Pipe: the server executes a script and throws the event as JSON to the standard input

• TPC and UDP: the server connects to a host and port and delivers the event

• AMQP: the event can be reinjected in the broker for further routing

Handling events selectively

• What if you don’t want to handle some events?

• REST API to the rescue

Stashes

• A stash is associated with an ID (a “path”)

• You can store here as much shit as you want

• Put a “silence” in the stash, and make your handlers check for it.

Mutators

• A filter to modify event data before passing it to a handler.

• Example: manipulate events to make them suitable for graphite before passing them to a TCP handler

Extensions

• Mutators and Handlers running in the same process than Sensu

• They must be written also in Ruby

• Run in the same event loop

Things missing / rant

• You can’t monitor something if you can’t install an agent on it.– There’s a ticket open since more than one year

ago...– Some patches, but still not a solution

• As with nagios, the state is decided by the plugins.

Further research

• http://sensuapp.org/docs/0.12

• https://github.com/sensu

• http://sensuapp.org/support/

• https://metacpan.org/pod/Sensu::API::Client