INNOV-4: Breaking Down Enterprise Application Data Silos with Data Services Ken Rugg Vice President,...

INNOV-4: Breaking Down Enterprise Application Data Silos with Data Services

Ken RuggVice President, Data Services

© 2006 Progress Software Corporation2INNOV-4 Breaking Down Data Silos with Data Services

Agenda

What are Data Services?…and why would I want them?

How does it work? The DataXtendTM Product Line

Breaking Down Enterprise Data Silos with Data Services


The Silos run the asylum

ERP

CRM SRMPLM

BI/DW

B2B

/ B

2C

B2B

CPM

Warranty

Asset Tracking

Vendor Compliance

BillingReservations


ETL Tools

Integration Brokers

Database APIs/Gateways

File Transfer/FTP

Virtual Data Federation ("EII")

Complexity and Breadth

of Applicability

Data Integration Landscape

Data Latency RangeNarrow Wide

DataTransformation

Support

Simplistic

Rich

Data Integration Forms the Technology Foundation of EIM, January 25th 2005

DataXtend Data Services

How is data delivered to applications today?

Replication

DataXtendTM

Data Services


DataXtendTM Data Services

DataXtendTM provides

Data integration for distributed, service-oriented applications.

Delivering:transactional views of shared data in a form that applications need in real timein real world environmentswith no change to your applications or business logic

Delivering Data to Applications



.NETJ2EEOpenEdge®

SQLServerOracleOpenEdge®OpenEdge®

OpenEdge®




Sonic Enterprise Service Bus

.NETJ2EEOpenEdge®


OpenEdge®






.NETJ2EEOpenEdge®


OpenEdge®



Data Services and SOA

“…we believe this idea of data services is a very important part of an overall approach to SOA.

Enterprises need to use data integration technology to create data services which

provide data to all these new and different consumers in a consistent, timely, high-quality,

and reliable manner.”

Ted Friedman, Gartner


Agenda





DataXtendTM: Addressing Business Challenges

Application versioning

Data integration

Master Data Management

Disconnected Applications

Service Oriented Architectures (SOA)

Optimized data access


ESBDelivering Data to Applications



DataXtendTM: Application Versioning

Description Side-by-side

deployment of old and new application

Database schemas differ

Benefits No change to apps Application & data

correctness Transparent data

conversion

App*App

DB DB*

Delivering Data To Applications



DataXtendTM: Data Integration

Description Integrate applications

by integrating their data Database schemas are

known

Benefits No impact to existing apps Application & data

correctness Transparent data

conversion

App 2App 1

DB 1 DB 2




DataXtendTM: Master Data Management

Description Multiple data sources Standard schema for

integrated data Complex process for

transforming data

Benefits Incremental change

capture Map data sources to

standard schema No impact to existing apps

App 3App 2App 1

DB 1 DB 3DB 2 Master DB

New app



DataXtendTM: Disconnected Operations

Description Enable mobile use of

database application Mostly connected or

mostly disconnected

Benefits No change to apps Application & data

correctness Transparent replication Efficient operation

DataXtend DataXtend Data ServicesDataXtend

MobileApp

Local DB

App

DB




.NETJ2EEOpenEdge®


OpenEdge®

DataXtendTM: Service Oriented Architectures

“Passing all the data that is required to complete a business process around as XML documents is like a snowplow pushing a snow bank down the street” – Carl Olofson, IDC


Each Service adds more data increasing the size of the “snow bank”



.NETJ2EEOpenEdge®


OpenEdge®



Each Service adds more data increasing the size of the “snow bank” By the end of the process the “snow can be very dirty” as the data becomes out of date






…with Data Services, information can be “passed by reference” to accelerate processing


.NETJ2EEOpenEdge®OpenEdge®

SQL ServerOracleOpenEdge®OpenEdge®


DataXtendTM

DataXtendTM: Optimized Data Access

Description Customized object view

of relational data Relational database is

system of record

Benefits Java object binding No change to legacy apps Transactional consistency In-memory performance

New appLegacy

app

DB of record

Legacyapp

DataCache





.NET

SQLServer

J2EEOpenEdge®

OracleOpenEdge®OpenEdge®

OpenEdge®

…in the context of SOA

SOA enables reuse of critical business functions as services




.NET

SQLServer

J2EE

Oracle


OpenEdge®

OpenEdge®OpenEdge®

OpenEdge® …But these reusable components can become

bottlenecks and single points of failure






.NET

SQLServer

J2EE

Oracle


Data Services enables “Optimized Services” with:• Optimized data model for specialized processing• Increased throughput and availability through

redundant services

OpenEdge®

OpenEdge®OpenEdge®

OpenEdge®



Agenda





DataXtendTM: How does it work?

Distributed transactional synchronization

Distributed Transaction Management• Similar to database clustering,

(e.g. Oracle RAC) What is a transaction

• An atomic unit of work with respect to– Consistency– Recovery

• Transactions provide– Atomicity– Correctness– Isolation– Durability

Before Execution

During Execution

After Execution

Consistent State

Potentially Inconsistent

State

Consistent State



Distributed transactional synchronization

What is a transaction (continued)• Serialization of updates

– If each transaction is correct, serial execution of these transactions is also correct

• Avoid conflicts with lock management

TXN 1

TXN 2

TXN 3

TXN 4

TXN 5

TXN 6



Distributed transactional synchronization Distributed Serialization

• Each replicated copy applies transactions in a common order• Avoid conflicts with distributed lock management

Similar to XA…• Distributed transactions• Strong transactional guarantees

…but different• All data goes every place

– each transaction can execute locally• Highly scalable, performant and available

TXN 1

TXN 2TXN 3

TXN 4TXN 5

TXN 6

TXN 1

TXN 2TXN 3

TXN 4TXN 5

TXN 6

TXN 1

TXN 2TXN 3

TXN 4TXN 5

TXN 6

TXN 1

TXN 2TXN 3

TXN 4TXN 5

TXN 6

TXN 1

TXN 2TXN 3

TXN 4TXN 5

TXN 6

Server 1 Server 3Server 2 Server 4 Server 5


Response Time: DataXtendTM vs. 2PC

0

5

10

15

20

25

30

1 Server 2 Servers 3 Servers 4 Servers 5 Servers

2 Phase Commit DataXtend Protocol

Response time in seconds of DataXtend Protocol vs. two phase commit across distributed servers



File System

Application

DatabaseTriggers

C. TablesS. Tables

Replication Engine

File Replicator

Distribution Manager

DataXtend™ RE DataXtend™ RE

DataXtend™ RE

DataXtend™ RE

~10-15% overhead for change detection

Transparent Change Detection




TABLE: CUSTOMER

ID NAME CITY STATE

114602992

100114105183108244200162200711

Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush

BostonSan JoseWichitaSt. LouisChicagoDallasButteErie

MACAKSMOILTXMTPA

TABLE: PROSPECTS

ID NAME CITY STATE

11451838244

Madeline JohnsonHenry OswaldRavi Kumar

St. LouisChicagoDallas

MOILTX

TABLE: CONTACTS

ID FIRST MIDDLE LAST ADDRESS

114602992

JoeMaryFrank

A.

Lee

SmithJonesAnderson

Boston, MASan Jose, CAWichita, KS

TABLE: CUST

ID NAME ZIP CITY

162711

Franklin, JonBush, Sid

5970116501

Erie, PAButte, MT

TABLE: ZIPCODE

ZIP

1650159701

CRM System

Customer Self-Service

Data Warehouse

Order Entry

Transactional Data Mapping




TABLE: CUSTOMER

ID NAME CITY STATE

114602992

100114105183108244200162200711



MACAKSMOILTXMTPA

TABLE: PROSPECTS

ID NAME CITY STATE

11451838244



MOILTX

TABLE: CONTACTS


114602992

JoeMaryFrank

A.

Lee

SmithJonesAnderson


TABLE: CUST

ID NAME ZIP CITY

162711


5970116501

Erie, PAButte, MT

TABLE: ZIPCODE

ZIP

1650159701

CRM System


Data Warehouse

Order Entry

Data Type Translation





TABLE: CUSTOMER

ID NAME CITY STATE

114602992

100114105183108244200162200711



MACAKSMOILTXMTPA

TABLE: PROSPECTS

ID NAME CITY STATE

11451838244



MOILTX

TABLE: CONTACTS


114602992

JoeMaryFrank

A.

Lee

SmithJonesAnderson


TABLE: CUST

ID NAME ZIP CITY

162711


5970116501

Erie, PAButte, MT

TABLE: ZIPCODE

ZIP

1650159701

CRM System


Data Warehouse

Order Entry

Format Translation





TABLE: CUSTOMER

ID NAME CITY STATE

114602992

100114105183108244200162200711



MACAKSMOILTXMTPA

TABLE: PROSPECTS

ID NAME CITY STATE

11451838244



MOILTX

TABLE: CONTACTS


114602992

JoeMaryFrank

A.

Lee

SmithJonesAnderson


TABLE: CUST

ID NAME ZIP CITY

162711


5970116501

Erie, PAButte, MT

TABLE: ZIPCODE

ZIP

1650159701

CRM System


Data Warehouse

Order Entry

Identity Translation





TABLE: CUSTOMER

ID NAME CITY STATE

114602992

100114105183108244200162200711



MACAKSMOILTXMTPA

TABLE: PROSPECTS

ID NAME CITY STATE

11451838244



MOILTX

TABLE: CONTACTS


114602992

JoeMaryFrank

A.

Lee

SmithJonesAnderson


TABLE: CUST

ID NAME ZIP CITY

162711


5970116501

Erie, PAButte, MT

TABLE: ZIPCODE

ZIP

1650159701

CRM System


Data Warehouse

Order Entry



Agenda





DataXtendDataXtend Data Services



Enables Data Integration as SOA enables Process Integration

OpenEdge®

OpenEdge®

SQLServer

.NET

Oracle

J2EE

OpenEdge®

OpenEdge®


DataXtend DataXtend Data Services




OpenEdge®

OpenEdge®

SQLServer

.NET

Oracle

J2EE

OpenEdge®

OpenEdge®

DataXtend


DataXtend DataXtend Data Services




OpenEdge®

OpenEdge®

SQLServer

.NET

Oracle

J2EE

OpenEdge®

OpenEdge®

DataXtend

DataCache


DataXtendTM Enterprise


.NETJ2EEOpenEdge®


OpenEdge®






.NETJ2EEOpenEdge®


OpenEdge®



DataXtendTM Remote Edition

Remote Office Operations•Support geographically dispersed operations of an enterprise

•Allows local office to continue operation when corporate network connection drops

Field Force Automation•Enterprise applications for users usually not connected to the corporate network

•User can continue to do business with no change in the quality of service of the application

Disconnected Operations


DataXtendTM Remote Edition

US Army Troop Deployment & Readiness SystemIssues: Information regarding troops did not give an

accurate picture of current units’ state of readiness Material asset readiness was old and outdated Battlefield conditions were not being captured and

delivered to where they were needed

Solution: Embed two-way Hub & Spoke replication in

FusionNet Data can be sliced for each field commander and his

specific troops

Results: All personal records, material assets, and troop

readiness can be viewed from the unit level

“DataXtendTM has been a great help in achieving our disconnected client and server replication goals”

Major Kurt WarnerCorps Knowledge

Management OfficerMulti-National Corps

Iraq, Office of the ACoS C6


DataXtendTM Caching Edition

DBDB DBDB DBDB DBDB DBDB

DBDB DBDB DBDB DBDB DBDB

Equities Trading

ServiceService ServiceService ServiceService ServiceService ServiceService

Enterprise Service Bus

40 global trading applications, $7B trades/day

Each application “re-invents” the data access layer:• Reduces developer productivity• Increases maintenance costs• Raises operating risks, system failures, downtime

DataXtendTM Caching EditionIntegrated Data Access And Caching





ESBDataXtendTM provides

Data integration for distributed,

service-oriented applications.

Delivering:transactional views of shared data in a form that applications need in real timein real world environmentswith no change to your applications or business logic


Questions?


Thank you foryour time

INNOV-4: Breaking Down Enterprise Application Data Silos with Data Services Ken Rugg Vice President,...

Documents

Transcript of INNOV-4: Breaking Down Enterprise Application Data Silos with Data Services Ken Rugg Vice President,...