Logz.io - TLV ELK Meetup #2 tlv (Summary of the PPT)
24
ELK Meetup – February 24 th 2015 Tomer Levy ([email protected] @Tomerlevy) Asaf Yigal ([email protected] @asafyigal)
-
Upload
tomerlevy9 -
Category
Software
-
view
216 -
download
1
Transcript of Logz.io - TLV ELK Meetup #2 tlv (Summary of the PPT)
ELK Meetup – February 24th 2015
Tomer Levy ([email protected] ֲ @Tomerlevy)
Asaf Yigal ([email protected] @asafyigal)
Is anyone using ELK to process logs?
Is anyone using the public cloud? AWS?
Is anyone doing kite-surfing?
Windsurfing?
Paragliding?
Sailing?
ELK – What is good for?
This is our office
And this is the view from the office
Our biggest challenge
When should we leave everything and go Kite-
Surfing?
Wind Analytics
Wind Across Locations?
Next Steps
Wind Analytics – next items
• Alerts
• Wind forecast combined with real wind
• Wind predictions!
Who Are We?
• ELK++ as a Service – Infinitely scalable
– Secured
– Highly Available
– Additional Features (Visual Grok, Alerts, role-based access…)
• Logz.io Insights: We will tell you what queriesare interesting for you– Machine Learning and Crowd Sourcing
How to monitor large AWS Environment?
• Some tips on how to create your own ELK deployment
• Shay Erlichmen @ Samba.me - demo
• A live demo of Kibana 4 implementation to monitor our internal deployment
ELK implementation – creating the right architecture
ELK implementation – creating the right architecture
Curator
Curator
3x Master Nodes + 1 data
ELK implementation – creating the right architecture
Curator
3x Master Nodes + 1 data
Index Failures Handler
ELK Basic implementation – find the weak spots
AZ-1
AZ-2
ELBLB
• Grok – parse logs to extract the relevant fields…
• Try our blog for some help on grok/plugins etc’
• blog.logz.io
ELK Basic implementation -Configuration
1.Use Elasticsearch AWS Plugin
2.EBS are challenging for big environment Use PIOPS if you can afford
3.Don’t run AWS cluster on the same AZ (but don’t run them on different zones!)
4.Use Shard allocation awareness
5.S3 Snapshots are cool! Things tend to break…
Elasticsearch Basic implementation -Configuration
Our architecture is very different – let’s look at it
DEMO
Let’s see a high level view of how we process log(z)
DEMO
Want to use the service?
• Feel free to email us for any question:
We’re Hiring – Big time!
• Elasticsearch Experts
• Java Developers
• Machine Learning Experts
• Graphic Designer
• Director of Marking
