PeerDirect Distributed Enterprise J. Espen Stokke alias “Curt…” Professional Services Manager.

PeerDirect Distributed PeerDirect Distributed EnterpriseEnterprise

J. Espen Stokke alias “Curt…”Professional Services Manager

© 2002 Progress Software Corporation2Sim

plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

PeerDirect Distributed EnterprisePeerDirect Distributed Enterprise

Introduction to Database Replication

Architecture Overview

Configuration

Replication Rule Design

Agenda


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Introduction to Database ReplicationIntroduction to Database Replication

Remote offices could benefit from local application performance, no matter where they are located?

Corporate data could automatically synchronize across the enterprise?

Your eBusiness infrastructure was infinitely scalable?

Remote users could update central databases when they are on the road?

What if…


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Introduction to Database ReplicationIntroduction to Database ReplicationSolve Common Expensive Problems . . .

Remote Users

Corporate Data CenterRemote Offices

How to Remote Restore?

Network Bandwidth and

Outages

Data Center Peak Performance

Remote Failover

Latency

Restricted Data Access

Linking Systems


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Integration - Multiple Production Site– No Designated Master

Bi-directional & Heterogeneous

– Two Way Update anywhere, propagate everywhere

Duplication - Single Production Site– Designated Master

Copy Source to Target(s)

– One Way Data Modifications only allowed at Source


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Back-up Fail-over Latency avoidance Load balancing Reporting Distributed environments

– Consolidation

– Synchronisation

Replication uses


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Log based (AI, Progress to Progress)– Database activity logged for periodic ‘replay’ at all

locations Queue based (SonicMQ, Any to Any)

– Middleware intercepts application to database activity for periodic replay at all locations

Table based (PeerDirect, Any to Any)– Captures changes, queries source and duplicates

values at all locations

Approaches


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Log based– Log size– Can not remain dormant (sleeping) long– Periodically need to re-synch– Inefficient use of bandwidth– Typically only hub and spoke topology

Queue based– All of the above– Application must be modified

Table based– Not real-time– Increases database size

Drawbacks of each approach


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess




Configuration


Agenda


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Architecture OverviewArchitecture OverviewFeatures and benefits

Feature Benefit Bi-directional update-

everywhere model Read-write data

between multiple databases

Replicate, synchronize, and distribute corporate data across multiple locations

ProductionOLTP

ProductionOLTP

ProductionOLTP

Order Processing

Site A

Order Processing

Site B

Order Processing

Site C

Distributed DBMS Management


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Feature Benefit Scheduled

synchronization Net change

compression Strong encryption Subset data using

‘slices’

Maximize network efficiency, reduce costs

Security– Controlled access to

data


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Feature Benefit Replicates between

different database types– MS SQL Server and

Progress

– Oracle and Progress Supported databases

– Progress

– Oracle

– SQL Server

– DB2

Share data in mixed environments

Consolidated view of corporate data


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Feature Benefit Multiple topologies

supported– Peer-to-peer

– Hub and spoke

– Load balance clustering

Provide new options for building scalable systems

Flexible configurations


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Feature Benefit Auto-discovery of

nodes within the replication network

Improve quality of service and system availability

Improved system administration

Allows mobile workers who are disconnected or have low-bandwidth limitations access to enterprise applications


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Feature Benefit Database level

configuration– Adds needed tables to

replicated database to manage replication

– Does not modify user-defined tables

Existing application does not need to be altered


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Architecture OverviewArchitecture Overview

Dynamic Data Slicing Architecture (DDSA)– Table partitioning - schema– Work set partitioning - query– Column level partitioning - fragment– Dynamic data migration

Dynamic Bandwidth-Managed Partner Selection (DBP)– Auto-discovery, auto-balanced– Avoids overloading any one server– Backbone, server and workstation algorithms

Collision Avoidance Methodology (CAM)– Record fragment management – related columns– Consistent resolution contracts– Rich API for custom resolution

Patented replication technology


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

System Tables41 tables ofsetup data

Control Tables1 table per

database table


Data Table “M” Rows

Database

Application

Control Table “M” Rows

PK

PK

Table-based


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Cost– Typically 15% to 25%

size increase of the overall database size

– Scales linearly and is predictable

Advantages– No user table

modifications– No primary key

restrictions– Predictable footprint

System and control table cost

System Tables

41 tables of setup data

Control Tables

1 table perdatabase table

+


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Architecture OverviewArchitecture OverviewChange capture methods

System Tables

Control Tables

Non-ProgressDatabase

Application

Triggers

System Tables

Control Tables

ProgressDatabase

Application

PDR

System Tables

Control Tables

Database

Application

Embedded


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


PDDE 6.1 Progress 9.1D Service Pack 5 DataDirect Driver 4.1

– Available via FTP site

System Requirements


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Principal components

PeerDirect Distributed Enterprise

ReplicationEngine(PDRE)

ReplicationDesigner(PDRD)

ReplicationAdministrator

(PDRA)

PeerDirect Distributed Enterprise(PDDE)


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Available licenses


SDK InnerEdge OuterEdge


OuterEdgeWorkstation


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Available licenses

InnerEdge Servers are “complete” PDRE sites that exist as part of your data center and possess a complete set of data for replication to other sites.


InnerEdge



plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Available licenses

OuterEdge Servers are PDRE sites installed in remote office locations that replicate with the central data center or peer databases.


OuterEdge



plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess




Configuration


Agenda


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

ConfigurationConfiguration

Communicates via SQL through ODBC The closer the Replication Engine is to the

database, the better the performance– Supports Win32 and Linux

Configuration of InnerEdge and OuterEdge servers affect the way in which sites will be replicated

Replication Engine


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

ConfigurationConfigurationPDRE Setup

DB

PeerDirect

DB

Machine A

DB

PeerDirect

DB

Machine A Machine B

DB DBPeerDirect

Machine A Machine B Machine C


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


DB PeerDirect

4GL App.

PeerDirectDB

InnerEdge Server

Sample configuration A

OuterEdge Server


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


DB PeerDirect

InnerEdge Server

Sample configuration B

4GL App.

PeerDirectDB

OuterEdge Server

DB PeerDirect

InnerEdge Server

DB PeerDirect

InnerEdge Server

4GL App.

PeerDirectDB

OuterEdge Server

4GL App.

PeerDirectDB

OuterEdge Server

4GL App.

PeerDirectDB

OuterEdge Server


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Aircraft Manufacturer– Aircraft maintenance application– Maintenance records were handled either on paper or in different

centralized databases– Maintenance issues cause commercial aircraft delays and military

readiness issues Solution: One PeerDirect InnerEdge Server and multiple OuterEdge

Workstations– Remote capabilities, work sets, and occasionally connected users– Aircraft maintenance data captured while aircraft is being maintained

resulting in significant cost savings

InnerEdge

Distributed Enterprise

OuterEdge

OuterEdge

OuterEdge

OuterEdge


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

ConfigurationConfiguration PeerDirect Replication Network


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess




Configuration


Agenda


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Replication Rule DesignReplication Rule Design

Generic– No auto-increment fields– Primary keys or candidate keys must be identified– PDUSER must be created with proper DBA permissions

Progress– Unique primary keys must be defined and available to

SQL engine Primary unique index

– 4GL dates must be SQL92 compliant– Be aware of column width overflow

Heterogeneous– Schemas of the data being replicated must match

Database considerations


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Specify the application database Select the tables to replicate Define fragments to minimize the number of data

collisions Subset data into work sets Arrange tables in transaction sets and/or groups Prepare to activate replication-enabled database Enable heterogeneous replication (if required)

Basic steps to defining rules


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

Replication Rule DesignReplication Rule DesignAnalyze the Schema

Order

PK,U1 Ordernum

FK1,FK2,FK3,U1 CustNumI1 OrderDate

ShipDatePromiseDateCarrierInstructionsPOTerms

I3 SalesRepFK3 BillToIDFK2 ShipToIDI2 OrderStatus

WarehouseNumCreditcard

RefCall

PK,U2,U1 CallNum

FK1,U1 CustNumCallDateSalesRep

U2 ParentI1 Txt

BillTo

PK,FK1 CustNumPK BillToID

NameAddressAddress2City

FK2 StatePostalCodeContactPhone

Salesrep

PK SalesRep

RepNameRegionMonthQuota

OrderLine

PK,FK1 OrdernumPK Linenum

FK2,I1 ItemnumPriceQtyDiscountExtendedPrice

I2 OrderLineStatus

ShipTo

PK,FK1 CustNumPK ShipToID

ContactAddressAddress2City

FK2 StatePostalCodePhoneCommentsName

Customer

PK CustNum

I3 NameAddressAddress2City

FK2 StateI2 Country

PhoneContact

FK1,I4 SalesRepI1 Comments

CreditLimitBalanceTermsDiscount

I2 PostalCodeFaxEmailAddress

Invoice

PK Invoicenum

FK2,I1 CustNumI2 InvoiceDate

AmountTotalPaidAdjustment

FK1,I3 OrderNumShipCharge

SalesRep Work SetPartial Sports2000

Schema


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Select the database type Enter the location of the

database– Non-production

instance is recommended for development

Enter the PDUSER user and password

Enter any optional ODBC parameters

Specify the application database


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Specify the name of the project

Enter the name of the network

Enter a name for the replication release

Enter the project information


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Select the Not Replicated tab

Choose the tables you wish to replicate– Multi-select tables by

holding down the Ctrl or Shift keys

Right click the group and select Replicate Table

Select the Replicate tab and view the tables that are now selected to replicate

Selecting tables to replicate


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Verify primary key or select candidate keys *– Under the Key column,

click on the field that is the primary key for the table

– From the drop down box, select the appropriate ordinal

Select field to be marked as a GUID field

Table structure


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Generic replication– Unit of replication is whole record– Changes occur at multiple locations

One user changes Addr One user changes Limit

– Can cause ‘False Collisions’

Fragments

Name Addr City Zip Ctry Pymt Rating Limit


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Common solution to generic replication– Unit of replication is each field– Changes occur at multiple locations

One user changes Addr and Limit One user changes Addr and Zip

– Can cause ‘Silent Data Corruption’

Fragments

Name Addr City Zip Ctry Pymt Rating LimitName Addr City Zip Ctry Pymt Rating Limit


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


PeerDirect solution to generic replication– Unit of replication is a group of fields– Changes occur at multiple locations

One user changes Addr and Limit One user changes Addr and Zip

– Helps avoid ‘False Collisions’– Optimizes the replication cycle

Fragments

Name Addr City Zip Ctry Pymt Rating Limit


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Common update responsibility Database size

– 12 bytes/fragment Frequency of replication Frequency of data being changed

– All columns in a fragment change Primary keys can not be part of fragments Customized conflict handling can be

created

Fragment considerations


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Determines the order in which tables are replicated– Parent tables must be replicated before child tables– Alphabetic order is used if no foreign key

relationships are defined Table relationships are defined from the bottom

level upward– Child to parent

Must be defined if work sets or transaction sets are to be used

Foreign Keys


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


The Designer will read the database's key constraints and display the foreign key relationships if defined within your database

Foreign Keys


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


PeerDirect allows you to define the business rules for sub-setting around base tables

Introduction to work sets

Cust

Prod

Price

HistCreditAcct

ADetl Trans

TDetl

Ctry

Off

HistCreditAcct

ADetl Trans

TDetl

Off Prod

Price


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Define the ‘Ctry’ base table Subscribe the site to its country

Defining work sets

142 - Bouchard

217 - W. Gates

330 - A. Palmer

401 - L. Haney

. . .

84006239

93050403

93072677

96193414

97005567

1 - Canada

2 - USA

3 - Australia

. . .

Ctry

Cust

Acct12 - New York

29 - Toronto

37 - Montreal

44 - Sydney

. . .

Off

550 - N. Peart

98717696

99656643

. . .

99700174


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

PeerDirect Replication Network

ProductionOLTP

ProductionOLTP

ProductionOLTP

Order Processing Site A

Order Processing Site B Order Processing Site C

Distributed DBMS Management


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess


Breaks dependency on centralized application architectures

Radically improves employee productivity with decentralized applications– Disconnected use

Creates an inherently resilient application architecture

Eliminates the latency and availability problems caused by the centralized model



plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

PeerDirect Replication Network


plify

you

r bu

sin

ess

Sim

plify

you

r bu

sin

ess

PeerDirect Distributed Enterprise J. Espen Stokke alias “Curt…” Professional Services Manager.

Documents

Transcript of PeerDirect Distributed Enterprise J. Espen Stokke alias “Curt…” Professional Services Manager.