Incremental Consistency Guarantees - USENIX · Incremental Consistency Guarantees Dragos-Adrian...

Incremental Consistency Guarantees For Replicated Objects

Rachid Guerraoui, Matej Pavlovic, Dragos-Adrian Seredinschi

Incremental Consistency Guarantees Dragos-Adrian Seredinschi

replicate

SOCIAL MEDIA APPLICATION

replicate

17Number of recent eventson the user’s timeline

• Returns the correct data • Latency: ~200 ms • Can become unavailable

Strong Consistency

[CAP], [PACELC]

replicate

Number of recent eventson the user’s timeline

• Latency: ~100 ms • High availability • Allows inconsistencies: can return

Strong Consistency

or or[CAP], [PACELC]

Weak Consistency

replicate

2022 17

Strong Consistency

Weak Consistency

replicate

2022 17

Neither model is ideal!

Strong Consistency

Weak Consistency

replicate

2022 17

Neither model is ideal!

We use both models.

Multiple models1. Weak consistency

2. Strong consistency

Multiple models1. Weak consistency

Dynamo [SOSP’07]

Pileus [SOSP’13]

Increasingly many systems exposemultiple consistency models:

Multiple models

1. Send multiple requests?

2. How to leverage individual responses?

3. Semantics?

4. …

1. Weak consistency

Issues

Dynamo [SOSP’07]

Pileus [SOSP’13]

Multiple models

How do you program with ★ inconsistencies? ★ multiple values?

Problem1. Send multiple requests?

2. How to leverage individual responses?

3. Semantics?

4. …

1. Weak consistency

Issues

Dynamo [SOSP’07]

Pileus [SOSP’13]

ABSTRACTION FOR REPLICATED OBJECTS

Consistency

Strong

Consistency

Strong

Consistency

…CORRECTABLE

Consistency

Strong

Consistency

…CORRECTABLE

Incremental Consistency Guarantees (ICG)

provides

Consistency

Correctables / Design

PROMISE

resolveasynchronously

➤ Starting point: Promises

➤ Placeholders for values

➤ Becoming mainstream

CORRECTABLEPROMISE

value1

value2

valuen

CORRECTABLEPROMISE

PROGRESSIVELY STRONGER CONSISTENCY

(INCREMENTAL)

PROGRESSIVELY HIGHER LATENCY

value1

value2

valuen

Consistency Models are Complementary

sisten

Strong

LowHigh PerformanceEventual

Linearizable

Sequential

Causal

(Ideal protocol)

(Worst protocol)

INCREM

Weak consistency: ★ Fast ★ (Often correct)

Strong consistency: ★ Slower ★ (Correct with certainty)Weak

Consistency

Strong Consistency

sisten

Strong

Linearizable

Sequential

Causal

(Ideal protocol)

(Worst protocol)

INCREM

Consistency

Strong Consistency

So what?

sisten

Strong

Linearizable

Sequential

Causal

(Ideal protocol)

(Worst protocol)

INCREM

Consistency

Strong Consistency

So what?Latency optimizations

Speculating with Correctables

read timeline

CORRECTABLE

Lower latency of strong consistency

Consistency

value1

Strong

Consistency

value2

read timeline

CORRECTABLE

Latency gap: ~100 ms

Consistency

value1

Strong

Consistency

value2

read timeline

Speculative execution

• value1 is often correct

• Speculatively execute any further stepse.g., prefetch dependent data

[Existential Consistency. SOSP’15][PBS. VLDB 5(8)’12]

Verify based on value2

CORRECTABLE

Consistency

value1

Strong

Consistency

value2

read timeline

Speculative execution

• value1 is often correct

• Speculatively execute any further stepse.g., prefetch dependent data

[Existential Consistency. SOSP’15][PBS. VLDB 5(8)’12]

Verify based on value2

CORRECTABLE

Strongly-consistent timeline

Fetch timeline items

200 ms

100 ms

Traditional operation:

200 ms

100 ms

Speculative operation with ICG:

value2

100 ms

value1

100 ms

200 ms

100 ms

value2

value1

matches

value2100 ms

value1

100 ms

200 ms

100 ms

value2

value1

matches

value2100 ms

value1

100 ms

100 msRe-fetch

Speculation case-study

➤ Application: Twissandra ➤ Workload generated via YCSB ➤ Clients in Ireland ➤ Geo-replication on Amazon’s EC2

VRGCAL

Speculation case-study

★ Advertising System— Speculation case-study

★ Ticket-selling System— Exploiting application semantics

★ Overheads evaluation& Optimizations

★ Latency gaps between consistencymodels

➤ Application: Twissandra ➤ Workload generated via YCSB ➤ Clients in Ireland ➤ Geo-replication on Amazon’s EC2

VRGCAL

check the paper

Decreasing latency of strong consistency

Workload A (50:50 read/write) Workload C (read-only)

What is the latency of the fetch_timeline() operation?

Workload B (95:5 read/write)

Baseline

Read using a quorum of 2/3 replicasvs.

ICG 1. Weak: Read with 1/3 replicas 2. “Strong:” Read with quorum of 2/3 replicas

Baseline

0 50 100 150 200 250 300

Workload A

C2 (R=2) CC2 (R=1,2)

0 50 100 150 200 250 300Throughput (ops/sec)

Workload B

0 50 100 150 200 250 300

Workload C

0 50 100 150 200 250 300

Baseline

0 50 100 150 200 250 300

Workload A

C2 (R=2) CC2 (R=1,2)

Workload B

0 50 100 150 200 250 300

Workload C

0 50 100 150 200 250 300

★ Latency decrease by 40%

★ Throughput drop by 6%

★ Same consistency model (2/3 replicas)

Baseline

The Correctables abstraction enables you to: 1. Leverage consistency models incrementally 2. Lower latency of strong consistency

CORRECTABLE

value1

value2

valuen

Incremental Consistency Guarantees

@adizere

Conclusion

backup slides

Speculation // Syntactic sugar

Legacy code vs. Correctables

semantics

execution details

Correctable

Legacy code vs. Correctables

semantics

execution details

Correctable

Overheads

➤ Cassandra

➤ YCSB workload, various configurations

➤ Client in Ireland

➤ Replicas in Virginia, Frankfurt, and Ireland

1.2 1.4 1.6 1.8

2 2.2 2.4

30 60 120

Workload A

CC2*CC2

#Total client threads

Latest distribution:Zipfian distribution:

+77% +15%

30 60 120

Workload B

+77% +15%

Overheads

➤ Cassandra

➤ YCSB workload, various configurations

➤ Client in Ireland

➤ Replicas in Virginia, Frankfurt, and Ireland

1.2 1.4 1.6 1.8

2 2.2 2.4

30 60 120

Workload A

CC2*CC2

+77% +15%

30 60 120

Workload B

+77% +15%

0 2 4 6 8

10 12 14 16

1 2 4 6 8 10 12Effic

ZooKeeper Correctable ZooKeeper

# Clients

500 tickets 1000 tickets

1 2 4 6 8 10 12# Clients

500 tickets 1000 tickets

➤ ZooKeeper queue implementation

➤ Wasteful implementation (by default)

➤ We were able to improve — negative overhead

500 elements 1000 elements

Exploiting application semantics

➤ Ticket selling application

➤ Implemented through a ZooKeeper queue

➤ Buy ticket = dequeue operation

Ticket 1 Ticket 2 … Ticket 500

update

ticket X

ticket Y | NULL

(weak)

(strong)

if ( X > threshold) confirm()else wait for strong

if (! NULL) confirm()

invokedequeue()

Exploiting application semantics

➤ Ticket selling application

➤ Implemented through a ZooKeeper queue

➤ Buy ticket = dequeue operation

Ticket 1

400 420 440 460 480 500La

Ticket number

Correctable ZooKeeper ZooKeeper

Last 20 tickets

Ticket 2 … Ticket 500

update

ticket X

ticket Y | NULL

(weak)

(strong)

if ( X > threshold) confirm()else wait for strong

if (! NULL) confirm()

invokedequeue()

Divergence between weak and strong consistency

10 15 20 25 30

30 60 120 180 240 300

Workload A-LatestWorkload A-Zipfian

Workload B-LatestWorkload B-Zipfian

➤ Extremely high

➤ Unusual

0.1% — 5% typical range}

Latency gaps between consistency models

100 150 200 250

0 200 400 600 800 1000

Workload A (50:50 read/write)C1 (R=1) C2 (R=2) CC2 preliminary (R=1) CC2 final (R=2)

100 150 200 250

0 200 400 600 800 1000Throughput (ops/sec)

100 150 200 250

0 200 400 600 800 1000

Workload C (read-only)

CC preliminaryCC final

C99th %ile latency

R=3 R=2 R=1

50 100

150 200

CZK preliminaryCZK final

ZK99th %ile latency

Leader in IRL Leader in VRG

Follower (FRK)

Leader (IRL)

Follower (IRL)

Leader(VRG)

Clientconnection:

value1

value2

Efficiency of Multiple Responses

CORRECTABLE

read timeline

Binding

Request

Replicated Storage

Response(final)

Response(preliminary)

Weak consistency Strong consistency

Coordination

value1

value2

CORRECTABLE

read timeline

Binding

Request

Replicated Storage

Response(final)

Response(preliminary)

Weak consistency Strong consistency

Coordination

value1

(quorum gathering)

value2

CORRECTABLE

read timeline

Correctables / Library

Application

Storage

binding binding binding binding

CorrectablesLIBRARY

invoke

(Weak / Strong)

stency-

DesktopApplication

WebFrontend

MobileApp

CachingDaemon

Correctable

Correctables / Library

Application

Storage

binding binding binding binding

CorrectablesLIBRARY

invoke

(Weak / Strong)

stency-

DesktopApplication

WebFrontend

MobileApp

CachingDaemon

Correctable

★ E-mail

★ Calendar

★ Social networks

★ Online shopping

★ Ad serving

★ News reading

★ Collaborative editing

★ Computation on static content

★ Cold data analysis

★ Disconnected operations in mobile applications

★ Infrastructure services

★ Stock tickers

★ Trading applications

Demand for Performance HighLowWeak

Strong

Weaker Consistency

Gray Zone

(no single choice is ideal)

Perform

ance is a

second-order

concern

Semantics allow

to bypass the need for strong

consistency

Replicated storage systems

Dynamo [SOSP’07]

Pileus [SOSP’13]

Richness of consistency models (CMs)

(weak and str

1 CM (st

rong) 10 CMs

Caching

(clien

t & se

CAP theorem

dynamic

Replicated storage systems

Dynamo [SOSP’07]

Pileus [SOSP’13]

Richness of consistency models (CMs)

(weak and str

1 CM (st

rong) 10 CMs

Caching

(clien

t & se

CAP theorem

No single consistency model is ideal → expose multiple choices

dynamic

Incremental Consistency Guarantees - USENIX · Incremental Consistency Guarantees Dragos-Adrian...

Documents

Transcript of Incremental Consistency Guarantees - USENIX · Incremental Consistency Guarantees Dragos-Adrian...

Consistency models: strict, sequential consistency

Combining client-centric and data-centric consistency models · The impact of data-centric models on session guarantees Session guarantees to obtain a data-centric model Data-centric

New Declarative Programming over Eventually Consistent Data Stores · 2020. 9. 1. · levels, with stronger consistency guarantees incurring increased latency and throughput costs.

INTRODUCTION - dfa.ms.gov · Web viewConfirm that all discount guarantees are direct savings off of AWP and not the result of incremental savings due to repackaging of prescriptions

MySQL ENTERPRISE EDITION · MySQL Group Replication provides native high availability with built-in group membership management, data consistency guarantees, conflict detection and

A Dynamic Data Replication with Consistency Approach in ... · consistency models (known as session guarantees), and data-centric consistency models. They used a consistent notation

The Distributed Object Consistency Protocol Version 1 · The Distributed Object Consistency Protocol Web Cache Consistency 2 2.1 Strong Consistency Models Under a strong consistency

``Single Point Incremental Forming and Multi-Stage Incremental ...

SPPA-T3000 System Overview...SPPA-T3000 eliminates these sub-systems. Its homogenous system architecture guarantees intrinsic data consistency and fewer components, resulting in a

Consistency and Replication. Outline Introduction (what’s it all about) Data-centric consistency Client-centric consistency Replica management Consistency.

EMS solutions tailored to your needs Brochure sp.pdf• Investment in the latest Fuji NXT SMT equipment ensures first-rate component placement accuracy. • This guarantees the consistency

Quality-of-Data for Consistency Levels in Geo-replicated ...lveiga/papers/cloud... · Spanner does use Paxos for strong guarantees on replicas. Strong consistency does not work well

Microsoft®...TLA+ specifications of the consistency guarantees provided by Cosmos DB Murat Demirbas [2018-11-01 Thu]

QoS Guarantees

Levels of Consistency Node Consistency (NC) Arc-consistency (AC) Path Consistency (PC) Generalised arc-consistency (GAC) Bounds consistency Inverse Path.

Incremental Load Development Method - …precisionrifleblog.com/.../07/incremental-load-development-method.pdf · Incremental Load Development Method Incremental Load Development

Consistency & Replication Consistency & replicationmcrane/CA464/Slides/slidesLecture10.pdf · Consistency & Replication Data-Centric Consistency Models Data-centric consistency models

MARMVS: Matching Ambiguity Reduced Multiple …...view depth map estimation with cross-view consistency. In BMVC, 2014. [38] Changchang Wu. Towards linear-time incremental struc-ture

We have a DREAM: Distributed Reactive Programming with Consistency Guarantees - DEBS 2014

Incremental Parsing by Modular Recurrent Connectionist ...papers.nips.cc/paper/201-incremental-parsing-by-modular-recurrent... · Incremental Parsing by Modular Recurrent ... Incremental