6.4 Data and File Replication

Gang Shen

Why replicate

PerformanceReliabilityResource sharingNetwork resource saving

Challenge

Transparency Replication Concurrent control Failure recovery Serialization

Atomicity

In database systems, atomicity is one of the ACID transaction properties. An atomic transaction is a series of database operations which either all occur, or all do not occur[1].

All or nothing

Atomicity

In DFS (Distributed File System), replicated objects (data or file) should follow atomicity rules, i.e., all copies should be updated (synchronously or asynchronously) or none.

Goal

One-copy serializability: The effect of transactions performed by clients on replicated objects should be the same as if they had been performed one at a time on a single set of objects.[2]

Architecture

FSA , File service agent, client interface RM, replica manager, provide replication

functions [3]

Architecture[3]

RM

RM

RM

RM

FSA

FSA

Client

Client

Read operations [3]

Read-one-primary, FSA only read from a primary RM, consistency

Read-one, FSA may read from any RM, concurrency

Read-quorum, FSA must read from a quorum of RMs to decide the currency of data

Write Operations[3]

Write-one-primary, only write to primary RM, primary RM update all other RMs

Write-all, update to all RMs Write-all- available, write to all functioning

RMs. Faulty RM need to be synched before bring online.

Write Operations

Write-quorum, update to a predefined quorum of RMs

Write-gossip, update to any RM and lazily propagated to other RMs

Read one primary, write one primary

Other RMs are backups of primary RM No concurrency Easy serialized Simple to implement Achieve one-copy serializability Primary RM is performance bottleneck

Read one, Write all

Provides concurrency Concurrency control protocol needed to

ensure consistency (serialization) Achieve one-copy serializability Difficult to implement (there will be failed

TM to block any updates)

Read one, Write all available

Variation of Read one, Write all May not guarantee one-copy serializability Issue of loss conflict in transactions

Loss of Conflicts

Assume to RMs, (a,b), object X,Y replicated to both.

Two transactions T1:R(X),W(Y),commit T2:R(Y),W(X),commit

Loss of Conflict

If Xa,Yb failed, transaction as follows T1:R(Xa),(Yb failed),W(Ya),commit T2:R(Yb),(Xa failed),W(Xb),commit There is no conflict since no object is shared.

Thus loss conflict. This can introduce error.

Read quorum,Write quorum

Version number attached to replicated object Highest version numbered object is the latest

object in read. Write operation advances version by 1 Write quorum > half of all object copies Write quorum+read quorum > all object

copies

Gossip Update

Applicable for frequent read, less update situations

Increased performance Typical read one, write gossip Use timestamp

Basic Gossip Update

Used for overwrite Three operations, read, update, gossip arrive Read, if TSfsa<=TSrm, RM has recent data, return it,

otherwise wait for gossip, or try other RM Update, if Tsfsa>TSrm, update. Update TSrm send

gossip. Otherwise, process based on application, perform update or reject

Gossip arrive, update RM if gossip carries new updates.

Causal Order Gossip Protocol[3]

Used for read-modify In a fixed RM configuration Using vector timestamps Using buffer to keep the order

Windows Server 2003[4]

Support DFS “State based, multi master” scheduled

replication Use namespace for transparent file sharing Use Remote Differential Compression to

propagate change only to save bandwidth

Continued[5]

If replication detects a conflict, last update wins. No merge files, but copies are kept for reference.

Reference

[1] Wikipedia; http://en.wikipedia.org/wiki/Atomicity[2] M. T. Harandi;J. Hou (modified: I. Gupta);"Transactions with

Replication";http://www.crhc.uiuc.edu/~nhv/428/slides/repl-trans.ppt [3] Randy Chow,Theodore Johnson, “Distributed Operating Systems &

Algorithms”, 1998[4] "Overview of the Distributed File System Solution in Microsoft

Windows Server 2003 R2";http://technet2.microsoft.com/WindowsServer/en/library/d3afe6ee-3083-4950-a093-8ab748651b761033.mspx?mfr=true

[5] "Distributed File System Replication: Frequently Asked Questions";http://technet2.microsoft.com/WindowsServer/en/library/f9b98a0f-c1ae-4a9f-9724-80c679596e6b1033.mspx?mfr=true