Dependable Systems (CSE 890), Thursday, 27th 2003

IRLIRLInteroperable Replication Logic:Interoperable Replication Logic:

A three-tier approach to FT-CORBA A three-tier approach to FT-CORBA InfrastructuresInfrastructures

Authors: R. Baldoni, C. Marchetti, and A. Termini

University of Rome “La Sapienza”

Presented by: SeyedMasoud Sadjadi

A Presentation for CSE 890 Course at

Department of Computer Science and Engineering

Michigan State University

Acknowledgement:Acknowledgement: University of Rome “La Sapienza” (IRL)

– Roberto Baldoni– Carlo Marchetti– A. Termini

OMG (CORBA) DOC Group & Lucent Technology (DOORS)

– Douglas Smith– Balachandran Natarajan

UCSB & Eternal System Inc. (Eternal)– Lousie Moser– Michael Melliar-Smith– Priya Narasimhan

Agenda:Agenda: MotivationMotivation Background IRL Approach IRL Performance Analysis Discussion and Concluding Remarks

Motivation:Motivation: FT-CORBA Limitations:

– Interoperability limitations

– No support for partitioned systems

Two-Tier Replication Limitations:

– Client/Server Synchrony

– Thick Client

Support for Legacy CORBA ORBs

– Client ORB Transparency

IRL Solution:IRL Solution: Introducing Replication Mid-Tier

– Separation of Concerns– Supporting Client/Server Asynchrony– Supporting Thin Client (Client Autonomy)

CORBA Portable Interceptors– Legacy Client ORBs Supporting PI can benefit

from FT-CORBA– Interoperable Using IIOP

Agenda:Agenda: Motivation BackgroundBackground IRL Approach IRL Performance Analysis Discussion and Concluding Remarks

CORBA Overview CORBA Overview (background)(background) CORBA is a distribution middleware specification by OMG that

– Provides a distributed object computing model.– Hides the communication details, marshalling and un-marshalling.

Elements:– Object, Servant, Server, Client, ORB, ORB, Interface, Stubs, skeletons, DII, DSI, and

Object Adapter

CORBA Architecture [CORBA-Overview]CORBA Architecture [CORBA-Overview]

Fault Tolerance Overview Fault Tolerance Overview (back. (back. cont.)cont.) Murphy’s Law of Fault Tolerance:

– “The only thing that is certain is that the system is going to fail.”

The best that we can do is to

– reduce the probability of failure .

– but not to zero.

FT-CORBA achieves fault tolerance using:

– Entity redundancy

– Fault detection

– Fault recovery.

FT-CORBA Architecture FT-CORBA Architecture (back. (back. cont.)cont.)

is_alive()

CORBA ORB CORBA ORBCORBA ORB

ReplicationManager

FaultNotifier

FaultDetector

Client

CServer

S1

Server

S2

LoggingMechanism

FactoryFault

Detector

RecoveryMechanism

LoggingMechanism

FactoryFault

Detector

RecoveryMechanism

LoggingMechanism

set_properties()

create_object()

notifications

fault reportscreate_object()

Tutorial on Fault Tolerant CORBA © Eternal Systems, Inc, 2000

FT-CORBA Overview FT-CORBA Overview (back. cont.)(back. cont.) Redundancy is the basis of fault tolerance Fault Types:

– Processor faults, Network faults, Operating System hangs, Memory leaks, No Software design errors

Object replication – Unit of redundancy in the FT-CORBA

Strong replica consistency– All of the replicas have the same state– Stateless replicas

Not an issue– Stateful replicas

Passive and Active styles of replication.

Replication Styles Replication Styles (background (background cont.)cont.) Passive Replication

– Algorithm: Only one replica processes each request Other replicas are available as backups if required

– Pros and Cons: Lower memory and processing costs Slower recovery from faults

Active Replication– Algorithm:

Several replicas process each request– Pros and Cons:

More memory and processing costs Fastest recovery from faults

Underlying mechanisms are the same for both

Active Replication Active Replication (background (background cont.)cont.)

Object

Eternal Eternal Eternal Eternal Eternal

Eternal Eternal Eternal

Clientinvokes a method of

Server A Server A

Server B

Reliable totally ordered multicast

STOP STOP

Duplicate invocationssuppressed

Reliabletotally orderedmulticasts forrequests and replies

Object Object Object Object

Object Object Object

Duplicate repliessuppressed

STOPSTOP

Tutorial on Fault Tolerant CORBA © Eternal Systems, Inc, 2000

Passive Replication Passive Replication (background (background cont.)cont.)

Eternal Eternal Eternal Eternal Eternal

Eternal Eternal Eternal

Clientinvokes a method of

Server A Server A

Server B

Reliable totally ordered multicast

Primaryreplica

Primaryreplica

Only primary replica of Server A executes the method

Reply returnedfrom primary replica of Server Bto primary replica of Server A

Only primary replicaof Server Bexecutes the method

Reliabletotally orderedmulticastfor state transfer

ObjectObject Object Object Object

Object Object Object

Tutorial on Fault Tolerant CORBA © Eternal Systems, Inc, 2000

Agenda:Agenda: Motivation Background IRL ApproachIRL Approach IRL Performance Analysis Discussion and Concluding Remarks

IRL ArchitectureIRL Architecture

IRL Architecture [IRL]

IRL Basic Architecture [IRL]

•Separation of Concerns•Client/Server Asynchrony•Client Autonomy•Interoperability

Replication in IRLReplication in IRL

Type Component Stateful Technique

Host Specific

Local Failure Detector

YesCold Passive

IRL Factory No Stateless

Domain Specific

ReplicationManager

Yes Hot Passive

FaultNotifier Yes Active

ObjectGroupHandler

YesHot

Passive(1)

[http://www.dis.uniroma1.it/~irl/details.htm]

IRL DeploymentIRL Deployment

Client-side:– Re-invocation – Re-direction– Req. unique id – Obsolete address

Server-side:– Object replication– Detecting fault– State transfer

[IRL]

Interoperable Object Group Interoperable Object Group ReferenceReference

Type_idNumber of

ProfilesIIOP Profile IIOP ProfileIIOP Profile Multiple

Components Profile

tag_group_ version

ft_domain_id

object_group_id

object_group_version

TAG_INTERNET_IOP

ProfileBody

IIOP Version

Host PortObject

KeyComponents

Number ofComponents

TAG_GROUPComponent

TAG_PRIMARYComponent

OtherComponents

Tutorial on Fault Tolerant CORBA © Eternal Systems, Inc, 2000

Three-Tier Replication ProtocolThree-Tier Replication Protocol

Scenario 1 [IRL]

Three-Tier Replication Protocol Three-Tier Replication Protocol (cont.)(cont.)

Scenario 2 [IRL]

Agenda:Agenda: Motivation Background IRL Approach IRL Performance AnalysisIRL Performance Analysis Discussion and Concluding Remarks

Performance AnalysisPerformance Analysis Testbed:

– 6 x Intel Pentium II 600– Windows NT– JDK 1.3.1– JacORB 1.3.21– ORBacus 4.1– 10Mbps LAN

FN Accuracy [IRL]Minimum percentage increment to apply to LFD heart beatingPerfect Fault Detection

Performance Analysis Performance Analysis (cont.)(cont.)

Experiment Parameters [IRL]

Basic Benchmarks [IRL]

Performance Analysis Performance Analysis (cont.)(cont.)

Stateless Replication Performance [IRL]

Performance Analysis Performance Analysis (cont.)(cont.)

Stateful Replication Performance [IRL]

Performance Analysis Performance Analysis (cont.)(cont.)

Stateful Replication Performance: Client Latency [IRL]

Performance Analysis Performance Analysis (cont.)(cont.)

Percentage Incidence of IRL Components [IRL]

Agenda:Agenda: Motivation Background IRL Approach IRL Performance Analysis Discussion and Concluding RemarksDiscussion and Concluding Remarks

Network OSNetwork OS

ORB

Network

Application Layer

Host Layer

DistributionLayer

ORB

CommonLayer

Middleware Layers by Schmidt Middleware Layers by Schmidt [Taxonomy][Taxonomy] Where are different approaches in the big picture?

DomainLayer

EternalTotem Isis

Orbix

Isis

Orbix

EternalTotem

IRL

DOORS

Electra Electra

OGS

IRL

DOORS OGS

Concluding Remarks:Concluding Remarks: Introducing Replication Mid-Tier

– Separation of Concerns– Client/Server Asynchrony– Client Autonomy

Using CORBA Portable Interceptors– Legacy Client ORBs Supporting PI can use FT-

CORBA– Interoperable Approach Using IIOP– Timeout is not implemented

Portable interceptors are passive

References:References: [IRL] R. Baldoni, C.Marchetti, A.Termini "Active Software Replication

through a Three-tier Approach", in Prooceedings of the 21st Symposium on Reliable Distributed Systems (SRDS'02), pp. 109-118, October 13-16, 2002 Osaka, Japan.

[CORBA-overview] http://www.cs.wustl.edu/ schmidt/corba-overview.html.

[Taxonomy] D. C. Schmidt, “Middleware for real-time and embedded systems,” Communications of the ACM, vol. 45, June 2002.

[PI] C.Marchetti, L.Verde, and R.Baldoni, “Corba request portable interceptors: A performance analysis,” in the 3nd International Symposium on Distributed Objects and Applications (DOA 2001), (Rome, Italy), Sept. 2001.

[Eternal] L. Moser, P. Melliar-Smith, P. Narasimhan, L. Tewksbury, and V. Kalogeraki, “The eternal system: an architecture for enterprise applications,” in the 3rd International Enterprise Distributed Object Computing Conference (EDOC’99), July 1999.

[CORBA] http://doc.ece.uci.edu/CORBA/formal/02-06-33.pdf