hotdog a TD tool for DD
21
Copyright ©2015 Treasure Data. All Rights Reserved. hotdog a TD tool for DD Yamashita, Yuu (@yyuu) Treasure Data, Inc. [email protected] March 11, 2015
-
Upload
treasure-data-inc -
Category
Engineering
-
view
2.684 -
download
0
Transcript of hotdog a TD tool for DD
Copyright ©2015 Treasure Data. All Rights Reserved.
hotdog a TD tool for DDYamashita, Yuu (@yyuu) Treasure Data, Inc. [email protected]
March 11, 2015
Copyright ©2015 Treasure Data. All Rights Reserved.
about:yyuu
• 2014/10 - Software Engineer at Treasure Data Inc. • Past: Infrastructure Engineer at Geisha Tokyo Entertainment Inc.
• OSS • pyenv (github: yyuu/pyenv) • xtc (bitbucket: yyuu/xtc) • jetty-nosql-memcached (github: yyuu/jetty-nosql-memcached) • botornado (github: yyuu/botornado) • sbt-extras (github: paulp/sbt-extras)
2
Copyright ©2015 Treasure Data. All Rights Reserved.
Today’s topic
• hotdog - yet another command line tool for DataDog
4
Copyright ©2015 Treasure Data. All Rights Reserved.
What’s DD?
5
Copyright ©2015 Treasure Data. All Rights Reserved.
What’s DD?
6
• System metrics • dd-agent
• Application metrics • fluent-logger • td-agent • fluent-plugin-metricsense
Copyright ©2015 Treasure Data. All Rights Reserved.
TD on DD
7
Copyright ©2015 Treasure Data. All Rights Reserved.
TD on DD
• We have *MANY* servers on DD • BIG DATA NEEDS PROCESSORS
• We have many DataDog monitors to detect failures • Send notification from DD to Slack and/or PagerDuty
8
Copyright ©2015 Treasure Data. All Rights Reserved.
TD ❤️ DD
9
Copyright ©2015 Treasure Data. All Rights Reserved.
Today’s topic
• hotdog - yet another command line tool for DataDog
10
Copyright ©2015 Treasure Data. All Rights Reserved.
TD 😱 PD
11
So, what is i-XXXXXXXX?
Copyright ©2015 Treasure Data. All Rights Reserved.
How to get node info of i-XXXXXXXX?
• AWS Management Console • DataDog Infrastructure List • Chef / Knife • …
12
Copyright ©2015 Treasure Data. All Rights Reserved.
My solution; hotdog
13
Copyright ©2015 Treasure Data. All Rights Reserved.
My solution; hotdog
• Search hosts from DD tags • Search hosts from instance-id • Support expressions on DD tags
• AND, OR, NOT • wildcard characters (‘*’, ‘?’, ‘[‘, ‘]’)
• Supports cooperation with pssh/GNU Parallel
14
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
15
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser • Get tag->hosts map from “GET /api/v1/tags/hosts”
16
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser • Get tag->hosts map from “GET /api/v1/tags/hosts” • Build host<->tag index on on-memory SQLite
17
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser • Get tag->hosts map from “GET /api/v1/tags/hosts” • Build host<->tag index on on-memory SQLite • Evaluate the expression
18
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works• Parse given expression by PEG parser • Get tag->hosts map from “GET /api/v1/tags/hosts” • Build host<->tag index on on-memory SQLite • Evaluate the expression • Print the results in specified format (text, json, yam, etc.)
19
small expression interpreter based on parslet
Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• https://github.com/yyuu/hotdog
20
Copyright ©2015 Treasure Data. All Rights Reserved.
WE ARE HIRING!
21
Check: www.treasuredata.com
