Lessons from Netflix Mesos Clusters
44
-
Upload
sharma-podila -
Category
Software
-
view
720 -
download
2
Transcript of Lessons from Netflix Mesos Clusters
![Page 1: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/1.jpg)
![Page 2: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/2.jpg)
![Page 3: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/3.jpg)
![Page 4: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/4.jpg)
![Page 5: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/5.jpg)
![Page 6: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/6.jpg)
81 Million subscribers worldwide and growing
Scale
![Page 7: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/7.jpg)
○
○
![Page 8: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/8.jpg)
![Page 9: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/9.jpg)
![Page 10: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/10.jpg)
○○
○
![Page 11: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/11.jpg)
○○
○
○
![Page 12: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/12.jpg)
![Page 13: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/13.jpg)
![Page 14: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/14.jpg)
![Page 15: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/15.jpg)
![Page 16: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/16.jpg)
![Page 17: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/17.jpg)
vs.
For long running stateful services
![Page 18: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/18.jpg)
![Page 19: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/19.jpg)
![Page 20: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/20.jpg)
![Page 22: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/22.jpg)
Fitness
Pending
Assigned
Urg
ency
N tasks to assign from M possible agents
![Page 23: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/23.jpg)
○
○
![Page 24: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/24.jpg)
MasterMasterMaster
Agent
![Page 25: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/25.jpg)
![Page 26: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/26.jpg)
ENI 1
ENI 2
![Page 27: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/27.jpg)
Compute focused Agent GroupASG-11: Active
Compute focused Agent GroupASG-10: Inactive
Network focused Agent GroupASG-21: Active
Network focused Agent GroupASG-20: Inactive
Fenzo autoscaling sets ASG’s “desired” count, honors “min” and “max”
AWS EC2 can rate limit API calls
![Page 28: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/28.jpg)
![Page 29: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/29.jpg)
![Page 30: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/30.jpg)
○○
○○
![Page 31: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/31.jpg)
○
○○○
![Page 32: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/32.jpg)
![Page 33: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/33.jpg)
+ <
![Page 34: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/34.jpg)
○○
○○
○
![Page 35: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/35.jpg)
○
○
○
![Page 36: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/36.jpg)
![Page 37: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/37.jpg)
○○
○
![Page 38: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/38.jpg)
![Page 39: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/39.jpg)
![Page 40: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/40.jpg)
“user”: … ,
“application”: … ,
“labels”: … ,
“Containers”: {
“C1”: { … }
},
“engine”: {
“jobspec”: { … }
}
![Page 41: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/41.jpg)
![Page 42: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/42.jpg)
○
○○
○
![Page 43: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/43.jpg)
○○
![Page 44: Lessons from Netflix Mesos Clusters](https://reader033.fdocuments.us/reader033/viewer/2022042520/58a9ab861a28ab9c758b57f3/html5/thumbnails/44.jpg)