Post on 26-Mar-2015
Data Services forService Oriented Architecture
in Finance
D. Britton Johnston
Chief Technology Evangelist
Copyright © 2005 Progress Software Corporation. All Rights Reserved.2
Agenda
Service Oriented Architecture Ideal for high performance trading systems
SOA requires enterprise data architecture Reliable, consistent and timely data
Trading system case studies demonstrate benefits of well thought-out
and executed data architecture for SOA
Copyright © 2005 Progress Software Corporation. All Rights Reserved.3
Case Example: Sell-Side Bank
DBDB DBDB DBDB DBDB DBDB
DBDB DBDB DBDB DBDB DBDB
Real-Time Trading Applications
ApplicationApplication ApplicationApplication ApplicationApplication ApplicationApplication ApplicationApplication
Enterprise Service Bus
40 global trading applications, $7B trades/day, over 5,000 txns/second
Each application “re-invents” the data access layer:• Reduces developer productivity• Increases maintenance costs• Raises operating risks, system failures, downtime
Data ServicesIntegrated Data Access And Caching
Copyright © 2005 Progress Software Corporation. All Rights Reserved.4
The Optimist’s View of SOA
Messaging Services- SOAP- XML- UDDI/LDAP
Looser coupling of common tasks Reuse at long last through shared services Eliminates tyranny of silos Everything just works™
SOA
Copyright © 2005 Progress Software Corporation. All Rights Reserved.5
Distribution Can Cause Bottlenecks
Shared Data
Apps share data cache, data silos can be out of sync
Each app requires separate data, all data must stay in sync
Data Data Data
DB DB
Check_Avail
Place_Order
Show_Status
Check_Avail
App
Place_Order Show_Status
DB DB DB DB
Business drivers: lower cost, higher flexibilityTechnology enablers: grid computing, web services
Copyright © 2005 Progress Software Corporation. All Rights Reserved.6
SOA Data Consistency Problem
Data Data
DB2
Check_Avail Place_Order
DB1
Item 3 = “out of stock”Item 3 = “in stock”
Nightly Sync
Data silos can cause inconsistent results
Copyright © 2005 Progress Software Corporation. All Rights Reserved.7
The Pessimist's View of SOA
Looser coupling of commonly performed tasks…– But, tighter consistency for commonly used data
Reuse at long last through shared services…– But, lengthier development time for shared services
Eliminates tyranny of silos…– But, lose application boundaries
Everything just works™…– But, Nothing ever works as advertised™
Copyright © 2005 Progress Software Corporation. All Rights Reserved.8
Agenda
Service Oriented Architecture Ideal for high performance trading systems
SOA requires enterprise data architecture Consistent and timely data
Trading system case studies demonstrate benefits of well thought-out
and executed data architecture for SOA
Copyright © 2005 Progress Software Corporation. All Rights Reserved.9
Requirements For Data Services
DB
AppAppApp
Cache Cache Cache
Data ServicesDistributed Caching
O-R Mapping
Replication
Functional Services
DB DB
Data Caching Services: stage data with app for performance and scalability
Data Replication Services: position data for distributed computing, high availability
Data Mapping Services: native language bindings for optimal performance
Copyright © 2005 Progress Software Corporation. All Rights Reserved.10
When To Worry: The 50/50 Rule
Ob
ject
Mod
el
50+
cla
sses
< 5
0 c
lass
es
Request Rate(Peak transactions/sec)
< 50 TPS 50+ TPS
Data-intensive
applications
Model-intensive
applications
Transaction-intensive
applications
Basic applications
Requires intelligent caching
Requires data services layer
Requires model-driven O/R mapping
Copyright © 2005 Progress Software Corporation. All Rights Reserved.11
Real-Time Data Services “Stack”
DB 1 DB 3DB 2
•Flexibility – bindings
•Performance –caching improves response time
•Scalability –cache replication enables scaling
•Availability–reliable sync improves app resilience
Data IntegrationVirtualized Database
C# AppJava AppC++ App
Compute GridDistributed Execution
Cache Cache Cache
Real-time Data ServicesDistributed Caching
Copyright © 2005 Progress Software Corporation. All Rights Reserved.12
The Iceberg Model For SOA
SOA Strengths• Loose task coupling• Reuse of shared tasks• Eliminate silos
Messaging Services
Data Services
Functional Services
Legacy Environment
SOA Data Gotchas• Data consistency• Data bottlenecks• Data availability
Copyright © 2005 Progress Software Corporation. All Rights Reserved.13
Agenda
Service Oriented Architecture Ideal for high performance trading systems
SOA requires enterprise data architecture Consistent and timely data
Trading System Case Study demonstrate benefits of well thought-out
and executed data architecture for SOA
Copyright © 2005 Progress Software Corporation. All Rights Reserved.14
Case Study: Sell-Side Bank Business Requirements
Project Requirements– Front & middle office equity trading: >40 global apps– High transaction volumes: >5,000 TPS, millions per day– High availability: max downtime from failure <30 seconds– High scalability: support 5x volume at minimal cost– Reference data usage: all apps share common
reference & order book data = huge potential for bottleneck Deployment Architecture
– Service Oriented Architecture: trading tasks exposed as shared functional services
– Progress Real-time Data Services: Java binding, mapping, replication, caching
– Grid Deployment: Unix Servers (>100 CPUs), Multi-site (US, Europe, Asia), Messaging Middleware, SQL Database
Copyright © 2005 Progress Software Corporation. All Rights Reserved.15
NY
Order Service
Reporting Service
NY
Exchange Service
Case Study: Real-time Data Services Architecture
A-L S-ZM-R
Real-time Data ServicesDistributed Caching, Mapping, Synchronization
Counterparty ServiceSecurities
Service
Counterparty ServiceSecurities
Service
Counterparty ServiceSecurities
ServiceOrder Book Service
NJ
Partitioned databases
• O/R Mapping• Caching• Replication
App examples• Trading desk, STP• Auto-exec engine• VWAP Pricing
Relational Databases
Vendor Feeds• Reuters• Bloomberg
Validation Workflow• Extract• Transform• Data cleanse• Change mgmt
Copyright © 2005 Progress Software Corporation. All Rights Reserved.16
Case Study: Benefits Achieved Scalability: grid data services infrastructure scaled to
$7B/day in trades (mainframe maint savings > $4m/yr) Availability: stateful failover between grid data services
caches helped cut failover time from 5 min to 30 sec Productivity: SOA delivered 50% productivity through
service reuse, required up-front resource (~30% of team) Grid Data Services: distributed caching required to “grid
enable” stateful SOA services to run in compute grid
Copyright © 2005 Progress Software Corporation. All Rights Reserved.17
ROI For SOA*
2x Developer productivity: reliable shared services should account for > 50% of new application functionality
3x maintenance productivity: systems deployed using SOA can be maintained with 75% fewer resources
2x “virtual data center” savings: distributed application deployment with centralized data storage (aka virtual data center) can achieve 40% capital cost savings, 30% annual operating cost savings vs traditional data centers
* Source: Progress customer case studies
Copyright © 2005 Progress Software Corporation. All Rights Reserved.18
SOA Data Architecture Roadmap
1. Start with data virtualization: create “golden master” data
2. Add data services: provide consistent language bindings, distributed caching
3. Migrate functionality to SOA: plan to invest 30% of dev resources into shared services
Consolidate SW
infrastructure: eliminate silos,
DBs (2+yrs)
Data Services forService Oriented Architecture
in Finance
D. Britton Johnston
Chief Technology Evangelist