Shameless Statements about Replication Rachid Guerraoui School of Computer and Communication...

Shameless Statements Shameless Statements about Replicationabout Replication

Rachid Guerraoui

School of Computer and Communication Sciences, EPFL

Joint ruminations with Eli Gafni (UCLA-MSR)

Shameless Statements about Replication Slide 2

Replication is all over the Replication is all over the placeplace

• Replicated databasesReplicated databases

• Reliable middleware

• Storage systems

• Non-blocking data structures

• Group communication

Highlight the important principles

(results/algorithms)

Deconstructing Deconstructing replicationreplication

A perspective on replicationA perspective on replication

“Here are my principles. If you don’t like them, I have

others”

Groucho Marx

A grain of saltA grain of salt

• For now, let’s For now, let’s • (1) ignore performance and focus on • (2) strong and general replication of

• (3) an object shared by 2 processes

ReplicationReplication

opAopA opAopA

opBopBopBopB

The players

The scheduler decides which process goes next

The processes take steps

The game

O’O’

opAopA opBopB

Fair agreement on the order Fair agreement on the order

opAopA

opBopBopAopAopBopB

O’O’

opAopA

opBopB

ConsensusConsensusSharedShared

memorymemory

opBopBopAopA

StatementStatement

• (1) Behind every (1) Behind every replicationreplication lie a lie a

consensusconsensus and a and a shared memoryshared memory

ConsensusConsensus

Consensus is impossible: FLPConsensus is impossible: FLP

Asynchronous shared Asynchronous shared

memory systemmemory system

p1p1 p2p2

Synchronous consensus is possibleSynchronous consensus is possible

Synchronous systemSynchronous system

Asynchronous systemAsynchronous system

<>Synchronous consensus is <>Synchronous consensus is possiblepossible

<>Synchronous system<>Synchronous system

Consensus is almost possibleConsensus is almost possible

-synchronous system-synchronous system

“One person is missing and the whole world seems depopulated”

Alphonse de la Martine

Consensus is possible iff Consensus is possible iff -synchrony-synchronyp1p1 p2p2

Bottom lineBottom line

• (2) Behind every consensus lies some -synchrony

-synchrony-synchrony

Consensus is almost possible

Consensus (primary)Consensus (primary)

SharedShared

memorymemory

Consensus (primary)Consensus (primary)

SharedShared

memorymemory

V1V1crashcrash

Consensus (primary-backup)Consensus (primary-backup)

V2V2V1V1

SharedShared

memorymemory

V1V1 V1V1

SharedShared

memorymemory

crashcrashV1V1

V2V2V2V2

SharedShared

memorymemory

V1V1 V1V1

Consensus (2PC)Consensus (2PC)

Shared memoryShared memory

V1V1 commitcommit

V2V2V2V2commitcommit

abortabort

Towards indulgent consensusTowards indulgent consensus

• Indulgence: tolerates arbitrarily long periods of asynchrony, i.e., tolerates any prefix

Indulgence

« He that is without sin among you, let him cast the first stone at her » John 8:3-11

Indulgence

Always preserves safety

Ensures liveness whenever possible

« While there is life there is hope » Cicero

Indulgent consensus Indulgent consensus (3PC)(3PC)

commit/abortcommit/abort commit/abortcommit/abort

• The processes dynamically exclude one suffix of a run, using a system oracle:

Indulgent consensus Indulgent consensus (3PC)(3PC)

A failure detectorA failure detector

Weakest failure detector

• The weakest failure detector for consensus -

• The weakest failure detector question translates into the smallest suffix set to be excluded

Weakest failure detectorWeakest failure detector

p1p1 p2p2

Shared memory assumptionShared memory assumption

Helps better understand consensus Helps better understand consensus results (FLP, FD, 2PC, 3PC)results (FLP, FD, 2PC, 3PC)

Needed anyway for replication (and indulgent consensus)

shared memoryshared memory

ABD (Snapshot)ABD (Snapshot)

QuorumQuorum

writewrite

readread

The many faces of quorumsThe many faces of quorums

Byzantine quorums

Failure detector quorums

Refined quorums

Probabilistic quorums

• (3) Behind every shared memory lies a quorum

• (1) ignoring performance And focusing on• (2) strong and general replication of• (3) one object shared by 2 processes

The engineerThe engineer

In real systems, we do care about In real systems, we do care about performanceperformance and we are happy with and we are happy with weak replicationweak replication

Much ado about nothing?

What about performance?What about performance?

Let’s move now to a message passing system with communication delays/rounds

Synchronous system with few failuresSynchronous system with few failures

Plan for the worst and hope for the bestPlan for the worst and hope for the best

How many synchronous rounds does consensus How many synchronous rounds does consensus need? need?

A shared memory system of n processes with 1 failure can simulate x rounds of a synchronous system with x failures

At least t+1

The inherent price of The inherent price of indulgenceindulgence

• How many synchronous rounds does an indulgent consensus need to decide with f failures?

A shared memory system of n processes with 1 failure can simulate x+1 synchronous rounds of an indulgent consensus algorithm with x failures

At least f+2

• For how long does a system need to be synchronous for indulgent consensus to terminate?

No clue…

• How many servers need to be correct in order for indulgent consensus to decide in x synchronous rounds?

Refined quorums

More about performance…More about performance…

•Disk accesses?

•Throughput?

What about weak replication? What about weak replication?

Is consensus necessary for weak replication?Is consensus necessary for weak replication?

If replicas would never need to agree on any state, they would not be called replicas

What is weak replication? What is weak replication?

The answer, my friend, is blowin' in the wind

Does ad-hoc replication need consensus?

We need consensus among 2 processes

Say we know the semantics of an object, e.g., a queue? (weaker than consensus)

• Does eventual replication need consensus?

• It does eventually..

• Does probabilistic replication need consensus?

• It does need randomized consensus..

What if What if

• We give up safety and let some of the replicas disagree?

• We might need set-agreement

• We give up liveness and ensure termination only if k processes are concurrent?

• We might need set-agreement

The future of replication

• What form of quorum (shared memory) does a set-agreement actually need?

• For how long does a system need to be synchronous for indulgent set-agreement to terminate?

The one slide to rememberThe one slide to remember

• (1) Behind every (1) Behind every replicationreplication lie lie

agreementagreement and and shared memoryshared memory

• (2) Behind every agreement lies -synchrony

• (3) Behind every shared memory lies a quorum

Or at least this oneOr at least this one

What about more processes?What about more processes?

f-process wait-free systemf-process wait-free system

n-process (f-1)-resilient systemn-process (f-1)-resilient system

Shameless Statements about Replication Rachid Guerraoui School of Computer and Communication...

Documents

Transcript of Shameless Statements about Replication Rachid Guerraoui School of Computer and Communication...

Ruminations Summer 2008

Marcos K. Aguilera Rachid Guerraoui Virendra J. Marathe ...

San FranciscoCoun - Attorney General of CaliforniaTelephone: (818) 843..:8435 lalawyer@lawyet.com LEE & GAFNI,LLP Robert Y.Lee(SBN213848) Adam 1. Gafni (SBN 230045) 3699 Wilshire Boulevard,

1 © R. Guerraoui Registers Prof R. Guerraoui Distributed Programming Laboratory.

Ruminations on process

Ruminations on the Work of Frederick Schauer

Ruminations on Brahms's Chamber Music

Ruminations Fall 2007

1 © R. Guerraoui The Limitations of Registers R. Guerraoui Distributed Programming Laboratory.

Ruminations on Arendt: Five Takes

1 © R. Guerraoui Implementing the Consensus Object with Timing Assumptions R. Guerraoui Distributed Programming Laboratory.

Elastic Transactions Pascal Felber Vincent Gramoli Rachid Guerraoui.

Ruminations on the History of Video Game Design

Ruminations on Terrorism, Expiation and Exposition

Andrei Giurgiu, Rachid Guerraoui, Kévin Huguenin, Anne ... · Computing in Social NetworksI,II Andrei Giurgiu a, Rachid Guerraoui , Kevin Huguenin´ a,1,, Anne-Marie Kermarrecb aEPFL,

Rachid Guerraoui , EPFL

Ruminations on Suretyship

Greetings from the ED Ruminations by John McGlynn

Generalized Universality - ics.forth.gr · R. Guerraoui, EPFL E. Gafni, UCLA . 2 . 3 Act 1 Act 2 Act 3 Classical Universality Modern Universality Generalized Modern Universality .

RACHID GUERRAOUI, PETR KUZNETSOV, MATTEO MONTI, … · 2019. 6. 14. · AT2: Asynchronous Trustworthy Transfers RACHID GUERRAOUI, EPFL PETR KUZNETSOV, LTCI, Télécom ParisTech, University