San Antonio DevOps: Fluentd
-
Upload
hart-hoover -
Category
Technology
-
view
83 -
download
0
Transcript of San Antonio DevOps: Fluentd
input&{&&&file&{&&&&&path&=>&"/var/log/http.log"&&&}&}&filter&{&&&grok&{&&&&&match&=>&[&"message",&"%{IP:client}&%{WORD:method}&%{URIPATHPARAM:request}&%{NUMBER:bytes}&%{NUMBER:duration}"&]&&&}&}
Fluentd Events
Time Tag
Record
passed from source or
adding a parsed time
message routing in fluentd
JSON
#&read&logs&from&a&file&&<source>&&type&tailpath&/var/log/httpd.log&format&apachetag&apache.access&&</source>
#&DOCKER&OMG&<source>&&&type&tail&&&format&json&&&path&/var/lib/docker/containers/ID/IDYjson.log&&&pos_file&/var/lib/docker/containers/ID/IDYjson.log.pos&&&tag&docker.container&&&rotate_wait&5&&&read_from_head&true&</source>
#&save&access&logs&to&MongoDB&&<match&apache.access>&&type&mongo&database&apache&collection&log&&</match>&
#&Post&to&IRC&<match&**>&&&type&irc&&&host&localhost&&&port&6667&&&channel&fluentd&&&nick&fluentd&&&user&fluentd&&&real&fluentd&&&message¬ice:&%s&[%s]&%s&&&out_keys&tag,time,message&&&time_key&time&&&time_format&%Y/%m/%d&%H:%M:%S&&&tag_key&tag&</match>
#&forward&other&logs&to&servers&&<match&**>&&type&forward&&&<server>&&&&&&host&192.168.0.11&&&&&&weight&20&&&&</server>&&&<server>&&&&&&host&192.168.0.12&&&&&&weight&60&&&&</server></match>
#&Send&logs&to&ElasticSearch&<match&**>&&&type&elasticsearch&&&logstash_format&true&&&host&localhost&&&port&9200&&&index_name&fluentd&</match>
https://github.com/treasure-data/chef-td-agent
Install with Chef
Vagrantfile & kitchen.yml included!
https://github.com/dmytro/fluentd-cookbook
Possibly Better
https://gist.github.com/hhoover/4fceb09148a73f45136a