Service-Oriented Data Architecture (SODA) Jon Jahren Produktsjef, Microsoft Norge...
-
Upload
martina-perkins -
Category
Documents
-
view
220 -
download
0
Transcript of Service-Oriented Data Architecture (SODA) Jon Jahren Produktsjef, Microsoft Norge...
Service-Oriented Data Architecture (SODA) Service-Oriented Data Architecture (SODA)
Jon JahrenJon JahrenProduktsjef, Microsoft NorgeProduktsjef, Microsoft [email protected]
AgendaAgenda
Natural forces in dataNatural forces in data
Data in a services worldData in a services world
Introducing SODAIntroducing SODA
SQL 2005 SODA featuresSQL 2005 SODA features
Wrap upWrap up
Wave 1Wave 1
Processor
Code
Data New Data
“Data Processing”
Wave 2Wave 2
Enough storage to Enough storage to keep some data keep some data “online”“online”Processor
CodeData
“OLTP”
Wave 3Wave 3Enterprise Process and Enterprise Process and Data models led to Data models led to “Enterprise “Enterprise Applications”Applications”
Enterprise Applications Enterprise Applications linked via linked via “Integration” (EAI)“Integration” (EAI)EAI
ERP
HR
The Data MonolithThe Data Monolith
Previous 30 years Previous 30 years trended towards trended towards data consolidationdata consolidation
SOA, connected SOA, connected systems, common systems, common schema, pulling schema, pulling data apartdata apart
Data ProcessingData Processing
OLTPOLTP
EAEA
InternetInternet
B2B ServicesB2B Services
DevicesDevices
SOASOA
Dat
a C
onve
rgen
ce
Dat
a C
onve
rgen
ce
Data D
istribution
Data D
istribution
Data in a SOA WorldData in a SOA World
Implications of SOA on dataImplications of SOA on dataBreaking of data monolithsBreaking of data monoliths
Data becomes more distributedData becomes more distributed
Recognition of data classesRecognition of data classesDifferent treatment for different dataDifferent treatment for different data
SOA SOA → Event Driven Data Systems→ Event Driven Data Systems
Loose couplingLoose couplingHeterogeneous systemsHeterogeneous systems
TimeTime
SecuritySecurity
Consistency vs. DistanceConsistency vs. Distance
Transactions take Transactions take DB’s from one DB’s from one consistent state to consistent state to anotheranother
Doing this across Doing this across space and time is space and time is hard!hard!
AttemptsAttempts2-phase commit2-phase commit
Optimistic CCOptimistic CC
ReplicationReplication
DataDataConsistencyConsistency
““Distance”Distance”SpaceSpaceTimeTime
AuthorityAuthorityConnectednessConnectedness
Resilience and CorrectnessResilience and Correctness
Challenge:Challenge:• System CorrectnessSystem Correctness• Distance:Distance:
• SpaceSpace• TimeTime• AuthorityAuthority• ImplementationImplementation
• Loose couplingLoose coupling
Approach:Approach:• Acknowledge Acknowledge Separation Separation
• AuthorityAuthority• TransactionTransaction• ImplementationImplementation
• Recognize data classesRecognize data classes• Reference dataReference data
Result:Result:• Service OrientedService Oriented Database Architecture Database Architecture (SODA) (SODA)
• Native Web ServicesNative Web Services• Service BrokerService Broker• Service Logic HostService Logic Host (CLR) (CLR)• Query NotificationsQuery Notifications
Order Processing circa 1950Order Processing circa 1950
PaymentPayment
CatalogCatalogUpdatesUpdates
OrderOrderFormForm
CatalogCatalog
OrderOrderAckAck
InvoiceInvoice
OrderOrderCopyCopy
PickPickListList
InventoryInventory
InventoryInventory
LedgerLedger
Reference DataReference DataResource DataResource DataActivity DataActivity DataService Interaction DataService Interaction Data
Reference Data DefinedReference Data Defined
Used to create service requestsUsed to create service requestsFormat interpretable by all partiesFormat interpretable by all parties
Anyone can read the catalogAnyone can read the catalog
Has a stable identifierHas a stable identifierMarch 2005 catalogMarch 2005 catalog
Referenced by service requestReferenced by service requestItem 23, Page 64, March 2005 catalogItem 23, Page 64, March 2005 catalog
Can be infinitely replicated and cachedCan be infinitely replicated and cachedDoesn’t need to be 100% consistentDoesn’t need to be 100% consistent
Might be able to order from March catalog on Might be able to order from March catalog on April 1April 1st.st.
CatalogCatalog
Activity Data DefinedActivity Data Defined
Lifetime generally bounded by Lifetime generally bounded by business activity (order, itinerary, business activity (order, itinerary, etc.)etc.)
May be retained for reporting, etc.May be retained for reporting, etc.
Format is private to serviceFormat is private to service
Generally keyed to business activityGenerally keyed to business activityBackorder for order #268Backorder for order #268
Can be partitionedCan be partitioned
Can change partitioning without Can change partitioning without moving datamoving data
(Ephemeral – evens out over time)(Ephemeral – evens out over time)
PickPickListList
Resource Data DefinedResource Data Defined
Very long lifetimeVery long lifetimeSKU’s, Customers, Accounts…SKU’s, Customers, Accounts…
Format and update private to serviceFormat and update private to service
Highly concurrentHighly concurrentOne item’s “stock on hand” modified by One item’s “stock on hand” modified by many ordersmany orders
Re-partitioning requires moving Re-partitioning requires moving entriesentries
InventoryInventory
Service Interaction Data DefinedService Interaction Data Defined
The only way services communicateThe only way services communicate
Format interpreted by all partiesFormat interpreted by all partiesEveryone can read and interpret the Everyone can read and interpret the “order form”“order form”
It may get lostIt may get lostWhether mail, e-mail, internet, etc.Whether mail, e-mail, internet, etc.
If it gets lost, you send another copyIf it gets lost, you send another copyThe copy better be an exact replica of the The copy better be an exact replica of the lost onelost one
OrderOrderFormForm
Observations on Data TypesObservations on Data Types
Different Service Data Classes Have Different Service Data Classes Have Different CharacteristicsDifferent Characteristics
NoReadOnly
None NoneYesNo: HasOpen Schema
NoReadOnly
None NoneYesNo: Has
Open Schema
YesVeryLow
ObjectPersistance
OptimisticMaybeYes
YesMay Be
HighCOM+
TransactionalPessimistic(Locking)
MaybeYes
ServiceInteraction
Reference
Activity
Resource
Private toService? XML?
Con-currency
Needs En-capsulation?
ObjectStyle
ConcurrencyApproach
Kind ofData
Deployment TopologiesSingle Tier & Client/ServerDeployment TopologiesSingle Tier & Client/Server
All data updated All data updated directlydirectlyService interaction Service interaction data implicitdata implicit
(Order Data comes (Order Data comes from form)from form)
All data treated All data treated alikealike
CatalogCatalog
OrderOrderCopyCopy
PickPickListList
InventoryInventory
LedgerLedger
Deployment TopologiesMulti-TierDeployment TopologiesMulti-Tier
Reference Data Cloned Reference Data Cloned & Cached& Cached
Activity Data “Tightly Activity Data “Tightly Coupled” to back endCoupled” to back end
Application connected Application connected to app server in a to app server in a “Session”“Session”
InventoryInventory
LedgerLedger
CatalogCatalogCacheCache
CatalogCatalogMasterMaster
OrderOrderCopyCopy
PickPickListList
Application ServerApplication Server
ReplicationReplication
CatalogCatalogCacheCacheCatalogCatalog
CacheCache
Deployment TopologiesFully Distributed Connected ApplicationDeployment TopologiesFully Distributed Connected Application
InventoryInventoryServiceService
LedgerLedgerServiceService
CatalogCatalogMaint.Maint.
ServiceService
OrderOrderServiceService
KittingKittingServiceService
CatalogCatalogUpdatesUpdates
OrderOrder
OrderOrderAckAck
InvoiceInvoice
OrderOrder
PaymentPayment
PaymentPayment
Reference Data Cached Reference Data Cached RemotelyRemotely
Activity Data Managed by Activity Data Managed by servicesservices
All communications via Service All communications via Service Interaction Data (Messages)Interaction Data (Messages)
Supports Heterogeneous Supports Heterogeneous ImplementationsImplementations
What Role Databases?What Role Databases?
Role of databases in SOA?Role of databases in SOA?Reference Data Cache ManagementReference Data Cache Management
Service Interaction (Message) Data Service Interaction (Message) Data HandlingHandling
Activity Data Management Activity Data Management
Connected Sessions Connected Sessions → Event Driven→ Event Driven
Need Service Oriented Database Need Service Oriented Database Architecture (SODA)Architecture (SODA)
ScalingScaling
Techniques differ with dataTechniques differ with dataReference data Reference data
Shared Scalable DatabaseShared Scalable Database
Snapshot replication Snapshot replication
Query notificationsQuery notifications
Activity data Activity data Dynamic partitioningDynamic partitioning
Data dependent routingData dependent routing
Service interaction data Service interaction data Inherently asynchronousInherently asynchronous
Resource data Resource data Static partitioningStatic partitioning
Peer-to-peer or bi-directional replicationPeer-to-peer or bi-directional replication
Storage strategiesStorage strategies
Techniques differ with dataTechniques differ with dataReference data Reference data
Master copy in central databaseMaster copy in central database
Replicas onlineReplicas online
Activity dataActivity dataKeep copies for activity tracking, trending, miningKeep copies for activity tracking, trending, mining
Service interaction dataService interaction dataKeep EXACT copies of request and responseKeep EXACT copies of request and response
For retry and recoveryFor retry and recovery
Sometimes a legal requirementSometimes a legal requirement
Resource dataResource dataDivide processesDivide processes
Updates can be asynch with other categoriesUpdates can be asynch with other categories
Session Oriented DatabaseSession Oriented Database
Input & Events Input & Events processed by processed by applicationapplication
Passive DataPassive Data
Little binding Little binding between data and between data and “logic”“logic”
Database “Application”Database “Application”
Database SessionDatabase Session
Database “Application”Database “Application”
Service Oriented DatabaseService Oriented Database
Service Logic Service Logic bound to:bound to:
ServicesServices
Query NotificationsQuery Notifications
EventsEvents
Service Logic Service Logic Triggered:Triggered:
By data changesBy data changes
By message arrivalBy message arrival
By timer firingBy timer firing
Data becomes Data becomes “active”“active”
EventsEvents
Service LogicService Logic
Service LogicService Logic
Service LogicService Logic
Database “Application”Database “Application”
SODA RequirementsSODA Requirements
Event DrivenEvent DrivenCommunication endpoint for servicesCommunication endpoint for services
Great host for service logicGreat host for service logicDecoupled from messaging (asynch)Decoupled from messaging (asynch)
Support logic inside or outside DB Support logic inside or outside DB processprocess
Needs to support reference and Needs to support reference and activity data managementactivity data management
Support service centric programming Support service centric programming modelmodel
What about App Servers?What about App Servers?
Some SODA requirements Some SODA requirements implemented in Application Servers…implemented in Application Servers…
Message BrokeringMessage Brokering
Complex Logic HostingComplex Logic Hosting
Web Service HostingWeb Service HostingWeb Service Comm. EndpointsWeb Service Comm. Endpoints
Web Service MechanismsWeb Service Mechanisms
Caching supportCaching support
Analogy between TP-Lite and Analogy between TP-Lite and traditional TPtraditional TP
TP-Heavy vs. TP-LiteTP-Heavy vs. TP-Lite
TP-LiteTP-LiteEarly to mid 1990’s DB Vendors add TP Early to mid 1990’s DB Vendors add TP features to databasefeatures to database
Thread pooling/managementThread pooling/management
Logic hosting (stored procedures)Logic hosting (stored procedures)
Communication ManagementCommunication Management
Transaction coordinationTransaction coordination
Became “Good Enough” for most Became “Good Enough” for most needsneeds
Same thing will happen with SODASame thing will happen with SODA
SQL Server 2005 SODA featuresSQL Server 2005 SODA features
Query notificationsQuery notificationsFor scaling out data & presentation cachesFor scaling out data & presentation caches
Reference data scalingReference data scaling
Native web servicesNative web servicesService EndpointService Endpoint
Service brokerService brokerService centric architectureService centric architecture
Reliable messaging with complete database Reliable messaging with complete database integrationintegration
Service logic hostService logic hostCLR IntegrationCLR Integration
Service Broker
Query Notifications for Web CachingQuery Notifications for Web Caching
Query Processor
Response
Response
Request
Request
Query/Subscription
Result
Notification
IIS / ASP.Net SQL Server 2005
Dispatcher
Native SOAP accessNative SOAP accessStandards based access to SQL Standards based access to SQL ServerServer
No client dependencyNo client dependency
Improved InteroperabilityImproved Interoperability
New “ENDPOINT AS HTTP” objectNew “ENDPOINT AS HTTP” objectConfigure connection infoConfigure connection info
Configure authenticationConfigure authentication
Expose Functions & SPsExpose Functions & SPs
Expose TSQL BatchesExpose TSQL Batches
http://server1/aspnet/default.aspxhttp://server1/aspnet/default.aspx
http://server1/sql/pubs?wsdlhttp://server1/sql/pubs?wsdl
KernelKernelModeMode
ListenerListener
Asynchronous, Distributed, Decoupled Asynchronous, Distributed, Decoupled programming environmentprogramming environment
Fully integrated into the database engineFully integrated into the database engine
Programming Model - DDL and DMLProgramming Model - DDL and DML
Integrated Management and DeploymentIntegrated Management and Deployment
Dependent SQL Server 2005 features – Events, Dependent SQL Server 2005 features – Events, Query Notifications, SQLiMailQuery Notifications, SQLiMail
Customer Usage, benefits: Enables new Customer Usage, benefits: Enables new scenarios – async SPs, Scalable distributed scenarios – async SPs, Scalable distributed multi-db applicationsmulti-db applications
TransactionTransaction
Tran
saction
Tran
saction
Stored proc runs onStored proc runs on Different transactionDifferent transaction Different threadDifferent thread Different securityDifferent security Different timeDifferent time
ClientClientAppApp
Persisted Dialog
LaunchedLaunchedStoredStoredProcProc
Services Live In The Databases
Services Live In The Databases
Ongoing work in the databaseOngoing work in the databaseEach Service “instance” is kept in a databaseEach Service “instance” is kept in a databaseState is kept in the databaseState is kept in the databaseMessages are stored in the databaseMessages are stored in the database
Routing to a databaseRouting to a databaseIncoming messages are kept in the databaseIncoming messages are kept in the databaseThe message is matched to the state and the The message is matched to the state and the computation is performedcomputation is performedRouting incoming web service messages means delivering to the Routing incoming web service messages means delivering to the correct databasecorrect database
TransactionTransaction
ServiceService
SharedSharedStuffStuff
PrivatePrivateStateState
TransactionTransaction TransactionTransaction
SQLCLR IntegrationSQLCLR IntegrationCLR Deeply IntegratedCLR Deeply IntegratedHosting layer provides Hosting layer provides coordination of:coordination of:
ThreadsThreadsMemoryMemoryGarbage CollectionGarbage Collection
DDL, Metadata, DDL, Metadata, SecuritySecurityAllows System Allows System ExtensionExtension
Scalar FunctionsScalar FunctionsTable Value FunctionsTable Value FunctionsProceduresProceduresTriggersTriggersData AccessData Access
WindowsWindows
SQL-OSSQL-OS
Host LayerHost Layer
SQL CLRSQL CLR
SQL EngineSQL Engine
ConclusionConclusion
Must consider classes of data in SOAMust consider classes of data in SOA
Databases will evolveDatabases will evolveFrom:From: Connected & session oriented Connected & session oriented
To:To: Event driven & service oriented Event driven & service oriented
TP-Heavy TP-Heavy → TP-Lite evolution will → TP-Lite evolution will repeat itself from App Server → SODArepeat itself from App Server → SODA
SQL 2005 is a SODA platformSQL 2005 is a SODA platform