My Elephant PowerPoint By: Caleb Anderson. skeleton African elephant Asian elephant.
Tracking multi-tenant resource usage with "White Elephant"
-
Upload
adam-faris -
Category
Technology
-
view
719 -
download
1
description
Transcript of Tracking multi-tenant resource usage with "White Elephant"
Tracking multi-tenant resource usage with "White Elephant”
Adam Faris LinkedIn
Why track usage?
– Use Hadoop to process logs– Creates small file problem for HDFS– WebHDFS + HAR = “Problem Solver”
Job History Logs
– Requirements– Provides Data Aggregation– Provides Dashboard– Open Sourced by LinkedIn Engineering
http://en.wikipedia.org/wiki/White_elephant
Failed Tasks
Reduce Shuffle Bytes
It can do more?
• Total task time• Total speculative time• CPU Hours • Plus more
• Helps determine capacity
• Github: – https://github.com/linkedin/white-elephant
• LinkedIn Open Source Projects: – http://data.linkedin.com/opensource/white-elephant
• LinkedIn is Hiring: – http://careers.linkedin.com
• Questions/Comments:– twitter: @opsmekanix