The Other Kind of Distributed System
17
-
Upload
datawire -
Category
Technology
-
view
1.605 -
download
1
Transcript of The Other Kind of Distributed System
● Founded in 2014● Spoken with dozens of organizations that have adopted
microservices at scale● Just hosted our Microservices Practitioner Summit with
speakers from Uber, Netflix, Facebook, Yelp, HubSpot, New Relic, and others -- check out videos online at microservices.com
● NodeJS is ubiquitous as a microservices framework as part of these orgs
Some problems are obviously distributed● Streaming video (Netflix)● Dispatching drivers (Uber)● Global pub/sub (Twitter)
The nature of the problem demands a distributed solution.
What if the problem isn’t inherently distributed...● Pet Shop● CMS● ...
But you need 1000 developers all working on the same codebase.
This is the world we live in today
● Software is eating the world...● Businesses live or die on their ability to quickly and
continuously deliver and adapt massive software platforms
Systems are becoming distributed not (only) because of the nature of the problem, but because of the nature of those solving it.
Microservices (Continuously Herding Distributed Cats)
Distributed Development
Distributed S
ystems
Microservices
Continuous U
ptime
Continuous Uptime Distributed Systems● Mechanical Failure● Distance
The system needs to continue to function despite the failure of any individual component.
The system needs to continue to function despite stale or unavailable information.
● Challenging, but many well known patterns exist
Distributed Development
● Coordination is impossible● Unintended consequences cause bugs
Small cross functional teams operating independently to deliver discrete parts that function together as a whole.
● Well defined APIs between parts● Strong versioning discipline● Strict overarching release cadence● Strong testing discipline
Distributed Development of Cont. Uptime Distributed Systems● Humans are the primary source of failures● Coordination is still impossible● APIs are distorted by distance● Dynamic factors are difficult to simulate● Testing is a harder
This changes the way we work.
● This requires a new methodology and a new set of tools.
Microservices Methodology
● Make it easier to predict consequences of change○ Smaller Changes
○ Frequent Deploys
○ Stronger versioning
○ Explicit Contracts
● Make it easier to recover from unintended consequence○ Visibility
○ Rollbacks/Fixes
○ Isolation
Methodology -> Tooling
● Change in unit of abstraction: library -> service● Abstractions are now expressed as contracts between
services
Dynamic languages in microservices● Yesterday:
○ Refactor rails/django/flask/nodejs monolith -> Java● Today:
○ Extend rails/django/flask/nodejs monolith with polyglot services
● Shift in language balance○ Strong typing within a language doesn’t help as much
What’s needed for microservices?● Quickly apply/rollback a small change
○ DevOps best practices (e.g., CD)
● Service Isolation● Service Availability
Building service isolation & avail
● Service isolation○ timeouts
○ defaults
○ circuit breakers
● Service availability○ dynamic load balancing
○ back pressure
Needs to be deployed in every service
Datawire Connect: isolation and availability across services
● 100% Open Source, Apache Software License
● Quark IDL○ Describe your API
● Datawire Connect library automatically integrates service isolation and availability○ client side load balancing○ timeouts○ circuit breakers○ backpressure (coming soon)○ context propagation (coming soon)○ rpc, async, eventing
● Quark DSL permits extending the DC library with additional behaviors● Support for NodeJS, Python, Java, (Ruby coming soon)
Demo
Summary
● Microservices requires service isolation and availability
● Building this from scratch is hard, but we make it easy
○ https://github.com/datawire/datawire-connect
○ Give us feedback (GitHub, Slack, email)
● Lots more information on microservices online at microservices.com, including videos
Distributed Development
Distributed S
ystems
Microservices
Continuous U
ptime
