Patterns for the Pragmatic Web 1 - Charles Sanders Peirce: ARISBE
Charles Young - Orchestration Patterns
-
Upload
karthik-vinayakamurthy -
Category
Documents
-
view
505 -
download
6
Transcript of Charles Young - Orchestration Patterns
Copyright Solidsoft 2007
BizTalk Server 2006BizTalk Server 2006
10 Most Useful 10 Most Useful Orchestration PatternsOrchestration Patterns
Charles YoungPrincipal Consultant
Copyright Solidsoft 2007
Top 6 Questions that kept me Top 6 Questions that kept me awake...awake... Do I have 10 patterns? Who says they are the most useful? What if I only have 6 patterns? What if I have 60 patterns? What about anti-patterns? Surely not another seminar on
convoys and scatter-gatherer...
Copyright Solidsoft 2007
AgendaAgenda
What are orchestrations, anyway? Convoying – of course! Common techniques for hard
problems Mapping models to implementation Dynamic approaches Building the Service Bus
Copyright Solidsoft 2007
What is a Pattern?What is a Pattern?
A normative example...• ...worthy of imitation• ...used as a guide, model or prototype of
a design we plan to implement
Copyright Solidsoft 2007
What is an Orchestration?What is an Orchestration?
“The automated arrangement, coordination, and management of complex computer systems, middleware, and services” • Wikipedia
(fount of all knowledge and truth) “A type of collaboration in which a
primary service directly invokes other services”
Copyright Solidsoft 2007
Isn’t this justIsn’t this justEnterprise Integration Patterns?Enterprise Integration Patterns? http://www.eaipatterns.com/
NO!
...if I hear "Scatter-Gatherer“ one more time...
Content-Based Router Message Filter
Dynamic Router Recipient List
Splitter Aggregator
Resequencer Composed Msg.
Processor Scatter-Gather
Routing Slip Process Manager
Message Broker
Copyright Solidsoft 2007
Are you sure?Are you sure?
Well, orchestrations implement message-based integration patterns…
…but the have a more fundamental purpose:
They:• Coordinate• Synchronise• Manage
Copyright Solidsoft 2007
An old debate...An old debate...
What’s the difference between orchestration and choreography?
Some say they are synonyms Some say they are antonyms
• centralised control vs. distributed collaboration
• process vs. protocol
Copyright Solidsoft 2007
A self-evident truth...A self-evident truth...
The term…
‘orchestration’
...is not as cool as the term
‘choreography’
Copyright Solidsoft 2007
Orchestration...Orchestration...
A constrained viewpoint• Discrete• Centralised• Process-orientated• Imperative
Conceptual model maps directly to controller processes/workflows
Copyright Solidsoft 2007
Choreography...Choreography...
An expansive viewpoint• Multi-agent• Distributed• Protocol-orientated• Collaborative
Conceptual model maps directly to de-coupled interaction between autonomous agents/services
Copyright Solidsoft 2007
Orchestration Orchestration Choreography!Choreography!
Orchestrations control interaction between services...
...but orchestrations are services too! You can choreograph your
orchestrations You can orchestrate the parts of your
choreography
Copyright Solidsoft 2007
So what is an Orchestration?So what is an Orchestration?
A style or type of service • Process controller/manager services
A specific technology operating in a specific context• The BizTalk Message Box • Correlation, transactions, compensation,
etc., etc.
Copyright Solidsoft 2007
Why not use web services?Why not use web services?
Once, it was easy to decide• No WCF / WF
Orchestrations:• Are service-orientated & message based• Are scalable & multi-threaded• Support advanced features• Are administered from a single point• Are created through model-driven
development
Copyright Solidsoft 2007
The #1 DifferentiatorThe #1 Differentiator
It’s the Message Box, stupid!
“Does the service need to communicate with other services & systems across a set of scalable, asynchronous, persisted message queues?”
Yes or No
Copyright Solidsoft 2007
BizTalk Server vs. ‘Dublin’BizTalk Server vs. ‘Dublin’
The great debate• ‘Dublin’ is free(?), BTS is not• ‘Dublin’ looks and feels like BTS
But consider...• ‘Dublin’ and BTS are service-orientated• BTS is also message-orientated
Messages are first-class citizens, even in orchestrations
It’s the Message Box, stupid!
Copyright Solidsoft 2007
Microsoft’s shorthand...Microsoft’s shorthand...
‘Dublin’:- ‘Application server’• “hosts … business logic and business
processes for use by … applications” (Wikipedia, again)
• The service is king BTS:- ‘Integration server’
• Facilitates interaction between diverse systems, technologies and protocols across internal and external networks
• The message is king
Copyright Solidsoft 2007
What’s the Top 1 Anti-Pattern?What’s the Top 1 Anti-Pattern?
In a WCF / WF / Dublin world...• Using orchestration where the Message
Box isn’t needed• Architecture should promote synergy
between BizTalk Server and WCF / WF More about this later
Copyright Solidsoft 2007
...And the Top 1 Pattern?...And the Top 1 Pattern?
You’ve guessed it!• Using orchestration where the Message
Box IS needed• Architecture should promote synergy
between BizTalk Server and WCF / WF
Copyright Solidsoft 2007
Is this a good Orchestration Is this a good Orchestration Pattern?Pattern?
Copyright Solidsoft 2007
CONVOYSCONVOYS
Copyright Solidsoft 2007
ConvoyingConvoying
The best-known orchestration patterns• http://msdn.microsoft.com/en-us/library/ms942189.aspx
• Used when messages are logically grouped into ‘streams’ or ‘sets’
Two dimensions:• Uniformity
are the messages all of the same type?
• Sequentiality Do the messages flow as a ‘stream’
Copyright Solidsoft 2007
Four Core Orchestration PatternsFour Core Orchestration Patterns
Sequential Parallel
Uniform Non-Uniform
Copyright Solidsoft 2007
Most common convoy pattern All messages share same
type• Orchestration port has single
operation
Uniform SequentialUniform Sequential
Msg 1
Msg 2
Msg 3
Msg 4
Msg 5
Start
End Use loops to simplify Arbitrary-sized convoys
• Detecting convoy ends• Using timers
Beware Zombies! Messages may be in-flight
when Delay triggers
Copyright Solidsoft 2007
Loops generally not needed• Different message types must
be received in the correct order
Messages have different types• Single orchestration port
has multiple operations
Non-Uniform SequentialNon-Uniform Sequential
Msg 1
Msg 2
Msg 3
Msg 4
Msg 5
Start
End
Copyright Solidsoft 2007
Non-sequential ConcernsNon-sequential Concerns
What if the convoy can be optionally started by more than one message?
What if the convoy messages arrive out of order?
Copyright Solidsoft 2007
ParallelParallel
Msg 1 Msg 2 Msg 3
Start
End
Uniform Parallel• Least common convoy pattern• Occasionally useful when
using different subscription rules for same message types
Any message may start the convoy• Set all Receive shapes to
‘Activate’
Msg 2 Msg 3
Copyright Solidsoft 2007
ConvoyingConvoying
Which is more fundamental in BTS?• Correlation or Convoying?
Trick question!• Convoying is the most fundamental
correlation mechanism BTS offers• Convoying is a fundamental MEP• It’s a Message Box / subscription thing
Orchestrations play a supporting role
Copyright Solidsoft 2007
ConvoyingConvoying
‘Activate’ subscription• Stored in the subscription table• Can be ‘parameterised’ for ‘convoy sets’
Convoy Sets• Correlation sets in the subscription system• Match a maximum of three promoted properties• Represented by Correlation Sets in orchestrations One-time only delivery generally needed• Beware Zombies!
Ordered delivery often needed
Copyright Solidsoft 2007
DECOMPOSE FOR COMPLEX DECOMPOSE FOR COMPLEX CORRELATIONCORRELATION
Copyright Solidsoft 2007
Consider the following....Consider the following....
What happens when you try to build?
Error: a correlation may be initialized only once
Copyright Solidsoft 2007
Basic RuleBasic Rule
If it won’t compile, decompose• Break out into separate orchestrations
Copyright Solidsoft 2007
UNDERSTANDING UNDERSTANDING CORRELATION SETSCORRELATION SETS
Copyright Solidsoft 2007
Correlation Sets...Correlation Sets...
...do three things• Store a set of property values for later use• Create subscription rule conditions on
Receive shapes• Promote properties on Send shapes
Subscription only uses promoted properties Sometimes, you don’t use the stored
values• That’s OK!
Copyright Solidsoft 2007
Correlation Sets and Direct BindingCorrelation Sets and Direct Binding
If you understand Correlation Sets... • remember decomposition
...and if you use direct, Message Box, binding...
You can implement (almost) any interchange pattern you want!• More on this later
Copyright Solidsoft 2007
SELF-TARGETED SELF-TARGETED ORCHESTRATIONSORCHESTRATIONS
Copyright Solidsoft 2007
Why, oh why...Why, oh why...
BizTalk Server has inbuilt support for non-deterministic control messages• Used when you don’t know if the target
recipient is listening• If a control message is not delivered, it is
quietly discarded• E.g., Acks and Nacks
But you can’t create your own control messages!!!
Copyright Solidsoft 2007
Where would I use such messages?Where would I use such messages?
Event-driven control of long-running processes• Events occur non-deterministically• Events need to be correlated onto
specific orchestration instances IF they are still running
Copyright Solidsoft 2007
One obvious approach...One obvious approach...
Add event consumer orchestrations• Always consume event messages,
regardless of running orchestrations• Subscribe by general type – don’t use
correlation• Optionally use never-ending uniform
sequential convoy loop to optimise performance
Copyright Solidsoft 2007
A more elegant approach...A more elegant approach...
Init a CS on Init a CS on Activate ReceiveActivate Receive
Init a second CS of Init a second CS of the same type on the same type on Send_1Send_1
Follow first CS on Follow first CS on ‘Receive Event’‘Receive Event’
Uses ConvoyUses Convoy One-shot event One-shot event
emitteremitter
Copyright Solidsoft 2007
...with looping......with looping...
Copyright Solidsoft 2007
The takeaway...The takeaway...
When handing non-deterministic, correlated event and control messages, passing those messages between different branches of a parallel shape will often get you out of trouble.
Copyright Solidsoft 2007
MASTER / SLAVEMASTER / SLAVE
Copyright Solidsoft 2007
Orchestrating your orchestrations...Orchestrating your orchestrations...
Remember, orchestrations are services as well.
Composition and aggregation... Re-use...
Master Slave 2
Slave 3
Re-usableOrchestration
Slave 1
Copyright Solidsoft 2007
Mapping logical design to physical Mapping logical design to physical implementationimplementation
Master
Slave 1 Slave 2
Slave 3 Slave 4
High-levelDesign
DetailedLogical Design
DetailedLogical Design
PhysicalDesign
PhysicalDesign
Copyright Solidsoft 2007
TraceabilityTraceability
Copyright Solidsoft 2007
Orchestration invocationOrchestration invocation
Call Orchestration• No MessageBox invocation• A stronger form of scoping• ‘out’ parameters• Limited to same host
Start Orchestration• Asynchronous via message box• Internal ‘RPC’ messages
Copyright Solidsoft 2007
Orchestration invocationOrchestration invocation
‘Direct Binding’• Message Box – i.e., ‘do it yourself’
Remember correlation sets
• Partner ports• Self-correlating ports
Copyright Solidsoft 2007
DYNAMIC PATTERNSDYNAMIC PATTERNS
Copyright Solidsoft 2007
Dynamic BizTalk Dynamic BizTalk – the unknown country– the unknown country Static BizTalk configuration
• Routing Subscription rules Bindings
• Addresses• Adapter selection• Transformation• Retry policy• Service Windows• Etc., etc.,
Copyright Solidsoft 2007
Dynamic FeaturesDynamic Features
Dynamic Send Ports Inbound / outbound port mapping XML Message Type resolution
• XMLReceive Pipeline BAM Interception
XLANG/s Transform
Copyright Solidsoft 2007
Real-world challengesReal-world challenges
I need to change the composition of by business processes rapidly (e.g., to support new campaigns, deal with supplier churn, etc.
I need to be able to change and version parts of my process orthogonally to other parts, within the context of long-running activities, without disruption.
Copyright Solidsoft 2007
What we need...What we need...
Agile orchestration composition Fine-grained orchestration versioning Policy-driven mediation and routing
• Endpoint resolution• Transformation• Tracking
...or more precisely...we need the tooling to make this easy
Copyright Solidsoft 2007
Dynamic BizTalkDynamic BizTalk
Enabled through intermediate componentsEnabled through intermediate components• Resolution components (ESB Guidance Toolkit)Resolution components (ESB Guidance Toolkit)• BAM interceptors (built-in)BAM interceptors (built-in)• Rules Engine (with vocabularies)Rules Engine (with vocabularies)
Consumer ProviderResolver
PoliciesPolicies
Copyright Solidsoft 2007
Agile VersioningAgile Versioning
Orch 1
Orch 2
Tax Calc v1
Orch 1
Orch 2
Tax Calc v1
Tax Calc v2
Process 1
Process 2
Long–Running Activities
Copyright Solidsoft 2007
Agile CompositionAgile Composition
MasterOrchestration
Order stock from contoso.com
CalculateLocal Taxes
SheduleContrator 1
Mail brochure for Campaign A
Order stock from northwind.com
Calculate newLocal Taxes
SheduleContrator 2
Mail brochure for Campaign B
Copyright Solidsoft 2007
MOVING ON MOVING ON TO SERVICE BUSESTO SERVICE BUSES
Copyright Solidsoft 2007
Building the Service BusBuilding the Service Bus
Common service interaction ‘fabric’ Distributed architecture Centralised administration Platform and technology agnostic
• Standards• Adapters and bridges
Dynamic configuration and deployment
Copyright Solidsoft 2007
BizTalk Server 2006 – Service BusBizTalk Server 2006 – Service Bus
Adapters
Receive Locations
Pipelines Pipelines
Maps MapsSubscription
Subscription
Adapters
‘On-Ramp’Services
‘Off-Ramp’ & RelayServices
OrchestrationServices
Copyright Solidsoft 2007
Using BizTalk Server with WCFUsing BizTalk Server with WCF
Technology ‘segments’ Common transports
• WCF bindings Service directory
• Resolution• Policies and administration
Dynamic messaging
Copyright Solidsoft 2007
BizTalk Server and WCF / WFBizTalk Server and WCF / WF
Copyright Solidsoft 2007
SummarySummary
What are orchestrations? Convoying & Correlation Handling Events Mapping to Logical Models Implementing Dynamic Patterns Building the Service Bus
Copyright Solidsoft 2007
Delivering Service-Orientated Delivering Service-Orientated Solutions on the Microsoft PlatformSolutions on the Microsoft Platform
Thank youThank you