Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M....
-
Upload
lizbeth-neal -
Category
Documents
-
view
232 -
download
4
Transcript of Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M....
![Page 1: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/1.jpg)
Distributed Programming and Consistency: Principles and Practice
Peter AlvaroNeil Conway
Joseph M. HellersteinUC Berkeley
![Page 2: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/2.jpg)
Part I: Principles
![Page 3: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/3.jpg)
Motivation
![Page 4: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/4.jpg)
Why are distributed systems hard?
UncertaintyIn communication
![Page 5: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/5.jpg)
Why are distributed systems hard?
We attack at dawn
??
![Page 6: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/6.jpg)
Why are distributed systems hard?
Wait for my signal Then attack!
Then attack!
![Page 7: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/7.jpg)
Why are distributed systems hard?
Attack!No, WAIT!
?
![Page 8: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/8.jpg)
Distributed systems are easier when messages are
![Page 9: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/9.jpg)
Distributed systems are easier when messages are
• Reorderable
![Page 10: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/10.jpg)
Distributed systems are easier when messages are
• Reorderable• Retryable
![Page 11: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/11.jpg)
Distributed systems are easier when messages are
• Reorderable• Retryable• Retraction-free
![Page 12: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/12.jpg)
(notes)
• Point to make: convergent objects are NOT retraction-free.
![Page 13: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/13.jpg)
Context: replicated distributed systems
Distributed: connected (but not always, or well)
Replicated: redundant data
![Page 14: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/14.jpg)
![Page 15: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/15.jpg)
Context: replicated distributed systems
Running example: a key-value store
put(“dinner”, “pizza”)Dinner = pizza
Dinner = pizza
Dinner = pizza
get(“dinner”)
“pizza”
![Page 16: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/16.jpg)
Context: replicated distributed systems
Distributed replication is desirableDistributed consistency is expensive
![Page 17: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/17.jpg)
Consistency?
Definitions abound: pick one.
but try to be consistent…
![Page 18: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/18.jpg)
What isn’t consistent?
Replication anomalies:
• Read anomalies (staleness)• Write divergence (concurrent updates)
![Page 19: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/19.jpg)
Anomalies
Stale reads
put(“dinner”, “pasta”)Dinner = pizza
Dinner = pizza
Dinner = pizza
get(“dinner”)
“pizza”
Dinner = pasta Dinner = pasta
Dinner = pasta
?
![Page 20: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/20.jpg)
Anomalies
Write conflicts
put(“dessert”, “cake”)Dessert = cake
Dessert = fruitput(“dessert”, “fruit”)
?
![Page 21: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/21.jpg)
Consistency
Anomalies witness replication.
A consistent replicated datastore rules out (some) replication anomalies.
![Page 22: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/22.jpg)
Consistency models
• Strong consistency• Eventual consistency• Weaker models
![Page 23: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/23.jpg)
Strong consistency
AKA ``single copy’’ consistency
Replication is transparent; no witnesses of replication
![Page 24: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/24.jpg)
Strong consistency
• Reads that could witness replication block• Concurrent writes take turns
![Page 25: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/25.jpg)
Strong consistency
Some strategies:
• Single master with synchronous replication– Writes are totally ordered, reads see latest values
• Quorum systems– A (majority) ensemble simulates a single master
• ``State machine replication’’– Use consensus to establish a total order over reads
and writes systemwide
![Page 26: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/26.jpg)
Strong consistency
Drawbacks:
Latency, availability, partition tolerance
![Page 27: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/27.jpg)
Eventual Consistency
• Tolerate stale reads and concurrent writes• Ensure that eventually* all replicas converge
* When activity has ceased and all messages are delivered to all replicas
![Page 28: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/28.jpg)
Eventual Consistency
Strategies:
• Establish a total update order off critical path (eg bayou).– Epidemic (gossip-based) replication– Tentatively apply, then possibly retract, updates as
the order is learned
![Page 29: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/29.jpg)
Eventual Consistency
Strategies:
• Deliver updates according to a ``cheap’’ order (e.g. causal). – Break ties with timestamps, merge functions
hmmmm
![Page 30: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/30.jpg)
Eventual Consistency
Strategies:
• Constrain the application so that updates are reorderable
Won’t always work.When will it work?
![Page 31: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/31.jpg)
Eventual consistency – more definitions
• Convergence– Conflicting writes are uniformly resolved– Reads eventually return current data– State-centric
• Confluence– A program has deterministic executions• Output is a function of input
– Program-centric
![Page 32: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/32.jpg)
Eventual consistency – more definitions
• Confluence is a strong correctness criterion– Not all programs are meant to be deterministic
• But it’s a nice property– E.g., for replay-based debugging– E.g., because the meaning of a program is its
output (not its ``traces’’)• Confluent => convergent
![Page 33: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/33.jpg)
Eventual consistency – more definitions
Confluent => convergent
Deterministic executions imply replica agreement
![Page 34: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/34.jpg)
Eventual consistency – more definitions
But convergent state does not imply deterministic executions
![Page 35: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/35.jpg)
(peter notes)
• EC systems focus only on controlling write anomalies (stale reads always fair game, though session guarantees may restrict which read anomalies can happen)
• EC systems are convergent – eventually there are no divergent states
• Deterministic => convergent (but not tother way)• Determinism is compositional: two deterministic systems
glued together make a deterministic system• Convergence is not compositional: two convergent systems
glued together do not necessarily make a convergent system (eg if the glue is NM)
![Page 36: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/36.jpg)
(peter notes)
• Guarded asynchrony – need to carefully explain the significance of this
• Essentially, confluent programs cannot allow one-shot queries on changing state (even monotonically changing)
• One-shot queries must be converted into subscriptions to a stream of updates– That way, we are guaranteed to see the last
update to a given lattice
![Page 37: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/37.jpg)
(joe notes)
• There’s stuff you can do in The storage layer…. Or you can pop up.
• Layer vs language• Sequential emulation at the storage layer• Crdts – state-centric attempt to achieve relaxed
ordering (object-by-object)• Then bloom • The programming model should match the
computation model
![Page 38: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/38.jpg)
Distributed design patternsfor eventual consistency
![Page 39: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/39.jpg)
ACID 2.0The classic ACID has the goal to make the application perceive that there is exactly one computer and it is doing nothing else while this transaction is being processed.
Consider the new ACID (or ACID2.0). The letters stand for: Associative, Commutative, Idempotent, and Distributed. The goal for ACID2.0 is to succeed if the pieces of the work happen:
At least once, Anywhere in the system, In any order.
- Pat Helland,Building on quicksand
![Page 40: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/40.jpg)
ACID 2.0
• Associative -- operations can be ``eagerly’’ processed
• Commutative – operations can be reordered• Idempotent – retry is always an option• Distributed – (needed a ``D’’)
![Page 41: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/41.jpg)
ACID 2.0
Instead of low-level reads and writes programmers use an abstract vocabulary of reorderable, retryable actions:• Retry – a mechanism to ensure that all
messages are delivered• Reorderability -- ensures that all replicas
converge
![Page 42: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/42.jpg)
Putting ACID 2.0 into practice
![Page 43: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/43.jpg)
Putting ACID 2.0 into practice
1. CRDTs– A state-based approach– Keep distributed state in data structures
providing only ACI methods
2. Disorderly programming– A language-based approach– Encourage structuring computation using
reorderable statements and data
![Page 44: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/44.jpg)
Formalizing ACID 2.0
• ACI are precisely the properties that define the LUB operation in a join semilattice
• If states form a lattice, we can always merge states using the LUB.
![Page 45: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/45.jpg)
C(v)RDTs
Convergent Replicated Datatypes
Idea: • represent state as a join semilattice. • Provide a ACI merge function
![Page 46: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/46.jpg)
CRDTs
Data structures:
1. Grow-only set (Gset)– Trivial – merge is union and union is commutative
2. 2PSet– Two Gsets – one for adds, the other for tombstones– Idiosyncrasy: you can only add/delete once.
3. Counters1. Tricky! Vector clock with an entry for each replica2. Increment @ replica I => VC[i] += 13. Value: sum of all VC values
![Page 47: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/47.jpg)
CRDTs
Difficulties:
Convergent objects alone are not strong enough to build confluent systems.
![Page 48: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/48.jpg)
Asynchronous messaging
You never really know
![Page 49: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/49.jpg)
Asynchronous messaging
A B
C sendA B
C
![Page 50: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/50.jpg)
Monotonic Logic
The more you know, the more you know.
![Page 51: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/51.jpg)
Monotonic Logic
A B
C
E
D
A
C
E
select/filter
![Page 52: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/52.jpg)
Monotonic Logic
A B
C project /map
f(A) f(B)
f(C)
![Page 53: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/53.jpg)
Monotonic Logic
A BC D E
B
D
B
Djoin / compose
F
![Page 54: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/54.jpg)
Monotonic Logic is order-insensitive
A B
C
E
D
A
C
E
![Page 55: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/55.jpg)
Monotonic Logic is pipelineable
A B
C
E
D
A
C
E
![Page 56: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/56.jpg)
Nonmonotonic Logic
When do you know for sure?
![Page 57: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/57.jpg)
Nonmonotonic Logic
A BC D E
B
D
set minus A B
CD
E
Retraction!
Retraction!
X
Y
Z
![Page 58: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/58.jpg)
Nonmonotonic logic is order-sensitive
A BC D E
B
D
set minus A
C E
![Page 59: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/59.jpg)
Nonmonotonic logic is blocking
A
set minus A ?
A?
![Page 60: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/60.jpg)
Nonmonotonic logic is blocking
A
set minusA?
``Sealed’’
![Page 61: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/61.jpg)
CALM Analysis
• Asynchrony => loss of order• Nonmonotonicity => order-sensitivity• Asynchrony ; Nonmonotonicity =>
Inconsistency
[…]
![Page 62: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/62.jpg)
CALM Analysis
• Asynchrony => loss of order• Nonmonotonicity => order-sensitivity• Asynchrony ; Nonmonotonicity =>
Inconsistency
?
``Point of Order’’
[…]
![Page 63: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/63.jpg)
Disorderly programming
An aside about logic programming:
In (classical) logic, theories are • Associative and commutative– Consequences are the same regardless of the
order in which we make deductions• Idempotent– Axioms can be reiterated freely
![Page 64: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/64.jpg)
Disorderly programming
An aside about logic programming:
In (classical) logic, theories are Associative, Commutative, and Idempotent because
Knowledge is monotonic:
The more you know, the more you know
![Page 65: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/65.jpg)
Disorderly programming
An aside about logic programming:
It is challenging to even talk about order in logic programming languages [dedalus].
Yet we can build …
![Page 66: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/66.jpg)
Disorderly programming
Idea: embody the ACID 2.0 design patterns in how we structure distributed programs.
Disorderly data: unordered relationsDisorderly code: specify how data changes over time
![Page 67: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/67.jpg)
Bloom
![Page 68: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/68.jpg)
Bloom Rules do |mes, mem| [mem.address, mes.id, mes.payload]end
multicast <~ (message * members)
![Page 69: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/69.jpg)
Operational model
![Page 70: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/70.jpg)
Time
Set(Union)
Integer(Max)
Boolean(Or)
“Growth”:Larger Sets
“Growth”:Larger Numbers
“Growth”:false true
![Page 71: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/71.jpg)
Time
Set(merge = Union)
Integer(merge = Max)
Boolean(merge = Or)
size() >= 5
Monotone functionfrom set max
Monotone functionfrom max boolean
![Page 72: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/72.jpg)
72
Builtin LatticesName Description ? a t b Sample Monotone
Functions
lbool Threshold test false a ∨ b when_true() ! v
lmax Increasing number
1 max(a,b)
gt(n) ! lbool+(n) ! lmax-(n) ! lmax
lmin Decreasing number
−1 min(a,b)
lt(n) ! lbool
lset Set of values ; a [ b intersect(lset) ! lsetproduct(lset) ! lset
contains?(v) ! lboolsize() ! lmax
lpset Non-negative set
; a [ b sum() ! lmax
lbag Multiset of values
; a [ b mult(v) ! lmax+(lbag) ! lbag
lmap Map from keys to lattice values
empty
map
at(v) ! any-latintersect(lmap) ! lmap
![Page 73: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/73.jpg)
73
Quorum Vote in BloomL
QUORUM_SIZE = 5RESULT_ADDR = "example.org"
class QuorumVote include Bud
state do channel :vote_chn, [:@addr, :voter_id] channel :result_chn, [:@addr] lset :votes lmax :vote_cnt lbool :got_quorum end
bloom do votes <= vote_chn {|v| v.voter_id} vote_cnt <= votes.size got_quorum <= vote_cnt.gt_eq(QUORUM_SIZE) result_chn <~ got_quorum.when_true { [RESULT_ADDR] } endend
Map set ! max
Map max ! bool
Threshold test on bool
Lattice state declarations
Communication interfaces
Accumulate votesinto set
Annotated Ruby class
Program state
Program logic
Merge function for set lattice
![Page 74: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/74.jpg)
Convergence – a 2PSet
![Page 75: Distributed Programming and Consistency: Principles and Practice Peter Alvaro Neil Conway Joseph M. Hellerstein UC Berkeley.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649cdf5503460f949a90f2/html5/thumbnails/75.jpg)
The difficulty with queries
``The work of multiple transactions can interleave as long as they are doing the commutative operations. If any transaction dares to READ the value, that does not commute, is annoying, and stops other concurrent work.’’ -- Pat Helland