Netflix's Journey to the Cloud: Lessons Learned from Netflix's ...
Netflix's Edge Gateway Using Zuul
-
Upload
spring-io -
Category
Technology
-
view
752 -
download
0
Transcript of Netflix's Edge Gateway Using Zuul
![Page 1: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/1.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zuul @ NetflixBy Mikey Cohen - Manager Cloud Gateway, Netflix
@moldfarm
1
![Page 2: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/2.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 2
![Page 3: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/3.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Global Streaming of TV Shows and Movies
3
![Page 4: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/4.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Over 42 Billion Hours of Streaming in 2015
4
![Page 5: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/5.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Over 83 Million Subscribers
In nearly every country5
![Page 6: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/6.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
The Gateway : From the Internet to Services in the Cloud
GatewayGatewayGATEWAY
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
6
![Page 7: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/7.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Our Gateway (Zuul) @ Netflix
• Handles most netflix.com hosts• More than 1000 device types
– Hundreds of permutations of protocols and device versions
• Fronted by over 50 elbs • Handling tens of billions of requests per day• 3 AWS regions• Over 20 production Zuul clusters• Fronting about 10 “Origin” systems
7
![Page 8: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/8.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
• Evolution• Scale• Success• Failure
Our Journey to Zuul
8
![Page 9: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/9.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Why Have a Gateway?
9
![Page 10: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/10.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Why have a Gateway?
10
![Page 11: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/11.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Isn’t (fill-in-the-blank) Good Enough?
11
![Page 12: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/12.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Netflix’s Idea of a Gateway●Deeply rooted in Microservice ecosystem●Dynamic routing logic●Deep Insights●Load balancing●Availability focused●Service protection●Quality assurance tool
12
![Page 13: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/13.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Evolving to a Gateway...Evolving to the Cloud
13
![Page 14: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/14.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Netflix’s Public API (2008)
14
![Page 15: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/15.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
...The world was a simpler place...
15
![Page 16: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/16.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
A Public Developer’s Gateway
• Simple static rule-based routing• API portal• Access Control• Request authentication (OAuth)• Throttling - request caps• Basic Monitoring and Analytics• Caching• 3rd Party managed and developed• Weeks and months development cycles
16
![Page 17: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/17.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Streaming Devices using public API
• Early Streaming Devices - 2009 – Windows Media Center– XBox– PS3
17
![Page 18: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/18.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Migration to AWS - 2010
• Apigee• Device traffic, not
public• Controlling DC ->
cloud migration• Running in AWS• Netflix controlled
18
![Page 19: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/19.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
2011 - Streaming Success / General Instability
• Massive Growth• Rapid device
expansion• Pushing AWS limits• Netflix Nascent in
AWS
19
![Page 20: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/20.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Instability makes way for Innovation
20
![Page 21: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/21.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Shortcomings of what we had
• Ecosystem Mismatch• Static configuration• Deployment time - Hours• Manual deployment - Error prone• Critical bugs go through vendor• Automated testing not possible
21
![Page 22: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/22.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Netflix’s Microservice Ecosystem
Netflix Platform (Karyon, Hystrix)
Data Pipeline (Kafka, etc)
Discovery (Eureka)
Real-time analytics(Mantis)
Monitoring(Atlas)
Dynamic Properties (ARCHAIS)
Deployment Pipeline
AWS
EC2 S3
CryptoAuthentication Database (Cassandra)
Micro Service(Zuul)
22
![Page 23: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/23.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Spring Platform Microservice Ecosystem
Spring Platform (Karyon, Hystrix)
Discovery (Eureka)
Monitoring(Atlas)
Dynamic Properties (ARCHAIS)
Deployment Pipeline
AWS
EC2 S3
Micro Service(Zuul)
23
![Page 24: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/24.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
2012 - Zuul Created
24
![Page 25: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/25.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zuul in a Nutshell
25
![Page 26: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/26.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Request Lifecycle of a Request
26
![Page 27: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/27.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Example Filterclass DeviceDelayFilter extends ZuulFilter { def static Random rand = new Random() @Override String filterType() { return 'pre' } @Override int filterOrder() { return 5 } @Override boolean shouldFilter() { return RequestContext.getRequest().getParameter("deviceType")?equals("BrokenDevice"):false } @Override Object run() { sleep(rand.nextInt(20000)) //Sleep for a random number of seconds between [0-20] }}
27
![Page 28: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/28.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What Zuul Got Us:
• Handle Edge Cases• React quickly• Service Resiliency• Move fast• Microservice Insights
28
![Page 29: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/29.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What Engineers Saw:• Handle Edge Cases
• React quickly!• Service Resiliency
• Move fast! ←• Microservice Insights
29
![Page 30: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/30.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zuul : Early Successes!
• Easy and convenient• Instant results• High adoption• Happy customers
30
![Page 31: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/31.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zuul - A Victim of Success
• Business logic in Zuul• Affects system
resiliency • Gateway team in
critical path
31
![Page 32: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/32.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Creating a Gateway Strategy
32
![Page 33: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/33.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Principles of Netflix’s Gateway Strategy • Creative Routing• Dynamic Routing• Delivery Focused• Traffic Shaping• React Fast • Insights
33
![Page 34: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/34.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Creative Routing - Subclusters with Purpose
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
34
![Page 35: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/35.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Red / Green Deployments
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
InstrumentedInstrumented
squeezesqueeze
35
![Page 36: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/36.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Developer Test Branches
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
InstrumentedInstrumented
squeezesqueeze
36
![Page 37: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/37.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Instrumented Clusters (Trickling traffic)
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
37
![Page 38: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/38.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Squeeze Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
38
![Page 39: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/39.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Targeted Routing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
39
![Page 40: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/40.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Service “Canarying”
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
40
![Page 41: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/41.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
“Sticky” Canary
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
41
![Page 42: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/42.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Failure Injection Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
42
![Page 43: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/43.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Degraded Experience Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
43
![Page 44: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/44.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Gateway Features of Deployment Pipeline
Gateway Features Completing the Continuous Delivery Pipeline
Version Control Build Unit
TestsFunctional TestingDeploy Service
Canary
Sticky Canary
Failure Injection Testing
Squeeze Testing
Instrumented Servers
Production Push
44
![Page 45: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/45.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Traffic Shaping
45
![Page 46: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/46.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
A Global Cloud Deployment
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Zuul
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Zuul
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Zuul
DB
US-West-2
US-East-1 EU-West-1
46
![Page 47: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/47.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Global Cloud Routing
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Zuul
Zuul
Zuul
US-West-2
US-East-1 EU-West-1
47
![Page 48: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/48.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
A Failing Region
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Zuul
Zuul
Zuul
US-West-2
US-East-1 EU-West-1
48
![Page 49: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/49.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Gateway routing to other regions
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Zuul
Zuul
Zuul
US-West-2
US-East-1 EU-West-1
49
![Page 50: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/50.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Attack Detection & Prevention
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
50
![Page 51: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/51.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Other Zuul Responsibilities @ Netflix• Top Level request context
– Geolocation– Cookie / Token decryption
• Authentication– Hand off to Authentication Services
• Request/Response “normalization”– Device specific “weirdness”
• Chunked Encoding• Header truncations• URL fixes
• Testing / Debug support– Verbose Headers
– Geolocation overrides– Error Context
51
![Page 52: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/52.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Smart Load Balancing
GatewayGateway
Gateway
Origin (API)
52
![Page 53: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/53.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Smart Load Balancing - Bad Nodes
GatewayGateway
Gateway
Origin (API)
53
![Page 54: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/54.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Gateway Backoff and Blacklists Bad Nodes
GatewayGateway
Gateway
Origin (API)
54
![Page 55: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/55.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zone Failure - Blacklist the Zone automatically
GatewayGateway
Gateway
Origin (API)
55
![Page 56: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/56.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
React Quickly - Runtime Filter changes
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
Runtime Filter Injection
56
![Page 57: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/57.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
A Room with a View - Insights
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
Insights
57
![Page 58: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/58.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What’s new on with Zuul
58
![Page 59: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/59.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Zuul 2!!
• Zuul 2.0– Netty (non-blocking, async)– RxJava chained filters– Coming to OSS soon.
59
![Page 60: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/60.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Why Zuul 2?
• 100M+ Persistent connections– Push notifications– Events
• Transport protocol agnostic– HTTP/2– Websocket– HTTP– etc..
60
![Page 61: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/61.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Async Non-blocking vs Blocking
61
![Page 62: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/62.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What’s Next?
• Auto-remediation • Gateway as a service
–Self-service dynamic routing / route validation
–Control APIs for special routing functions
62
![Page 63: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/63.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Top Lessons Learned
63
![Page 64: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/64.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Ask Why?
64
![Page 65: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/65.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Build for handling Failures
65
![Page 66: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/66.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Take Advantage of your Microservice Ecosystem
66
![Page 67: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/67.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Use Routing Creatively
67
![Page 68: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/68.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Shard to Reduce Blast Radius
68
![Page 69: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/69.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Devices are WeirdProtocols are Weird
69
![Page 70: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/70.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Devices are ForeverProtocols are Forever
70
![Page 71: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/71.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Keep Business Logic out of your Gateway
71
![Page 72: Netflix's Edge Gateway Using Zuul](https://reader033.fdocuments.us/reader033/viewer/2022051300/586fdb301a28ab18428b6021/html5/thumbnails/72.jpg)
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Learn More. Stay Connected.
Mikey Cohen - @moldfarm
https://github.com/Netflix/zuulhttp://techblog.netflix.com
@springcentralspring.io/blog
@pivotalpivotal.io/blog
@pivotalcfhttp://engineering.pivotal.io
72