Fluentd and PHP

PHP & Fluentd

@chobi_e

chobi_e is: PECL Sundown Author (will be release soon) h>ps://github.com/chobie/php-‐sundown

libgit2 PHP Binding Author h>ps://github.com/libgit2/php-‐git

phpredis contributor

h>ps://github.com/nicolasff/phpredis

Scaling web servers.

Scaling web servers. basically, just add web server to the array.

then, how about collecLng log files?

using log file

using log file Are you serious?

using syslog like middleware

using syslog like middleware Not Bad.

using reliable and distributed log collector:

Note: fluent-‐logger-‐php does not support output buffering. So local fluentd as a buffer in this case.

using reliable and distributed log collector:

that’s the best soluLon IMHO

Note: fluent-‐logger-‐php does not support output buffering. So local fluentd as a buffer in this case.

fluentd provides robust messaging soluLon

Data Mining

for Data Mining

Data Mining

Queued job

for queued job

fluentd first step with PHP

install by composer

# composer.json { "name": "my-‐project", "version": "1.0.0", "require": { "fluent/logger": "master-‐dev" } }

wget h>p://getcomposer.org/composer.phar php -‐d detect_unicode=0 composer.phar install

<?php require "vendor/.composer/autoload.php"; use Fluent\Logger\FluentLogger; $logger = FluentLogger::open( "debug.test", "localhost", "24224” );

install by copy the tree.

git clone h>ps://github.com/fluent/fluent-‐logger-‐php.git cp -‐r src/Fluent <path/to/your_project>

php library which using Fluentd.

Monolog\Fluent by nrk h>ps://github.com/nrk/monolog-‐fluent.git

what is Monolog? Monolog by Seldaek. h>ps://github.com/Seldaek/monolog Monolog is: heavily inspired by Python's Logbook library, although most concepts have been adjusted to fit to the PHP world.

install $ wget h>p://getcomposer.org/composer.phar $ php composer.phar install

<?php use Nrk\Fluent\Monolog\FluentHandler; $log = new Monolog\Logger('debug.monolog'); $log-‐>pushHandler(new FluentHandler('h>p://127.0.0.1')); $log-‐>pushHandler(new FluentHandler('tcp://127.0.0.1')); $log-‐>addError("OH NOES!!11!1!");

introduce my fluent plugin

fluent-‐plugin-‐delayed -‐ delayed output plugin. h>ps://github.com/chobie/fluent-‐plugin-‐delayed

fluent-‐plugin-‐delayed -‐ delayed output plugin. h>ps://github.com/chobie/fluent-‐plugin-‐delayed

delayed plugin

event{ Lme: 2011-‐10-‐22 10:00:05, # 5 seconds future ... }

Client

: aler 5 seconds. then delayed plugin outputs the data to other output plugin.

output

other Ideas. fluent-‐plugin-‐cooldown -‐ coolodowned outputs when specified key received. fluent-‐plugin-‐sampler -‐ only outputs N % data.

GitHQ / Kokuban Github / gist clone (sLll under development)

updaLng ssh key is pain in ass. /home/git/.ssh/authorized_keys

web server fluentd cooldown exec client

redis update authorized_keys

send public key

sending message via fluentd\Logger

only pass cooldowned message.

updaLng SSH Authorized keys

obtain ssh public keys

Web developer only knows tag for queued job.

Web developer only knows tag for queued job.

Server engineer only implements reliable consumer plugin for Fluentd.

Sounds good?

fluent-‐logger-‐php status.

fluentd protocol support ✔ h>p protocol support ✔ composer support ✔ PEAR package support ✔ -‐ but I don’t provide pear channel.

message pack support buffering support -‐ does not support as there’s trouble some for using persistent variables for buffer.

That’s all

That’s all i need more developer! please feel free to pull req me. h>ps://github.com/chobie

Fluentd and PHP

Technology

Transcript of Fluentd and PHP