Graylog2 Syslog with Rails and MongoDB candy

Manage your logs in the dark and have lasersgoing and make it look like you're from space

Lennart Koopmann, 2010

About me

Lennart Koopmann22 years oldLiving in Hamburg, working at Jimdo

lennartkoopmann.net / @_lennart

phpLogCon let's not talk about thatGraylog1: August 2009Graylog2: August 2010

Syslog daemon

(TCP/UDP)

GELF

(UDP)

Graylog extended log format

UDP - Max (GZIP) 8192 byte per message(Chunking is supported)

{ }

{ 'message':'Exception: Something went wrong.' }

{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars' }

{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19' }

{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php' }

{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php','line':2638 }

{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php','line':2638, 'level':1 }

Chunking

Use Cases

Plain syslog

Collect everything of your /var/log from all servers, aggregate, analyze and get a warning if something goes wrong

GELF

Embed it into the logging class of your application and trigger with every error. Include stacktrace and env vars in full message. Later filter by file:line to get statistics.

Aggregate, analyze and get a warning if something goes wrong

Notifications

Currently only Nagios hook (rake task) Triggered at too many messages in last X minutes

Fine grained notifications coming with incident management. (Email, SMS APIs, XMPP)

Average page generation time

Graylog1 / MySQL: ~12000ms

Average page generation time

Graylog1 / MySQL: ~12000msGraylog2 / MongoDB: ~250ms

Roadmap

Incident management

Define chained rules that explain an incident. Trigger several actions when incident occurs.

Fine grained notifications

Be informed via Email, XMPP, SMS by Graylog2 directly without Nagios hook

More analyzing

Thank you.

www.graylog2.org / @Graylog2www.lennartkoopmann.net / @_lennart

(Slides are on Slideshare)