Constructing Software For Service Oriented Architecture

Jean-Jacques Dubray, Ph.D.Architect

jeanjadu@attachmate.com

Lecture, 03/26/2004The Pennsylvania State University

The Smeal College of Business Administration

© attachmate 2004

Copyright Notice

According to US and Worldwide Copyright laws, it is forbidden to use all or part of this presentation without a written consent of Attachmate

In particular, you are not allowed To remove attachmate logos or the author’s name Change the title of the presentation Publish part or all of the presentation under your

name or the name of another organization

© attachmate 2004

Acknowledgments

This presentation was reviewed and commented by Jeff Schneider, CEO of OpenStorm Eric Newcomer, CTO of Iona Paul Brown, CEO of Fivesight Ben Gaucherin, CTO of Sapient

I greatly appreciate their feedback

© attachmate 2004

Outline

Service Oriented Society The “connected” world From Component Orientation to Service

Orientation The Road to SOA

Three key concepts in software construction Conclusion

The Service Oriented Society

Imagine if we had to do everything

we need to get done by ourselves?

© attachmate 2004

From Craftsmen to Service Providers

Our society has become what it is today through the forces of Specialization Standardization Scalability

It is now almost exclusively “service” oriented Transportation Telecommunication Retail Healthcare Financial services …

© attachmate 2004

Attributes of physical services

Well defined, easy-to-use, somewhat standardized interface Self-contained with no visible dependencies to other services (almost) Always available but idle until requests come “Provision-able” Easily accessible and usable readily, no “integration” required Coarse grain Independent of consumer context,

but a service can have a context New services can be offered by combining existing services Quantifiable quality of service

Do not compete on “What” but “How” Performance/Quality Cost …

© attachmate 2004

Context, Composition and State

Services are most often designed to ignore the context in which they are used It does not mean that services are stateless they

are rather context independent ! This is precisely my / the definition of “loosely

coupled” Services can be reused in contexts not known at design

time Value can be created by combining, i.e. “composing”

services Book a trip versus book a flight, car, hotel, …

© attachmate 2004

Service Interfaces

Non proprietary All service providers offer somewhat the same

interface Highly Polymorphic

Intent is enough Implementation can be changed in ways that do not

break all the service consumers Real world services interact with thousands of

consumers Service providers cannot afford to “break” the context

of their consumers

© attachmate 2004

Intents and Offers

Service consumer expresses “intent” Service providers define “offers”

Sometimes a mediator will: Find the best offer matching an intent Advertise an offer in different ways such that it

matches different intent Request / Response is just a very particular

case of an Intent / Offer protocol

© attachmate 2004

Service Orientation and Directories

Our society could not be “service oriented” without the “Yellow Pages”

Directories and addressing mechanisms are at the center of our service oriented society

Imagine Being able to reach a service just by using

longitude and latitude coordinates as an addressing mechanism?

Only being able to use a service if you can remember its location, phone or fax number?

© attachmate 2004

Service Orientation is scalable

Consumers can consume and combine a lot of services since they don’t have to know or “learn” how to use a service

Service providers can offer their services to a lot more consumers because by optimizing The user interface Access (Geographical, Financial, …) They were able to provide the best quality of

service and optimize their implementations

© attachmate 2004

So…

Service orientation allows us Complete freedom to create contexts in which

services are uses and combined Express intent rather than specific requests

Our society should be a great source of inspiration to design modern enterprise systems and architectures or understand what kind of services these systems will consume or provide

The connected (new) world

Over the past 15 years,

everything got connected to everything else

© attachmate 2004

Connectivity Drives the Emergence and Convergence of Technologies

WorkflowWorkflow

EDIEDI

MainframeMainframe

?BusinessIntegrationBusiness

Integration

J2EE.NETJ2EE.NET

Client / Server

Web/Portal

EAIEAI

B2BB2B

BPMBPM

WSWS

OfficeOffice

1980 1990 2000 2010

XMLWS

WebLANInternetSOA

© attachmate 2004

Connectivity Enables Global Processes and Information Access

Information

BusinessProcesses

Local

1980 1990 2000 2010

Web

XMLWS

WAN

Web

LAN

LANInternet

Global

SOA

© attachmate 2004

Seamless Connectivity enables “On Demand” Computing

Use software as you need Much lower setup time, forget about

Installation Implementation Training Maintenance

Scalable and effective usage of resources Provision Billed on true usage Parallelism (CPU, Storage, Bandwidth…)

© attachmate 2004

But Seamless Connectivity is also questioning all our beliefs…

An application is NOT a single system running on a single device and bounded by a single organization

Continuum Object … Document Messages and Services

As opposed « distributed objects » Exchanges becomes peer-to-peer

Asynchronous communications Concurrency becomes the norm while our

languages and infrastructures did not plan for it

© attachmate 2004

…we are reaching the point of maximum confusion

Federation and Collaboration As opposed to « Integration »

Language(s) Semantic (not syntactic) Declarative and Model driven (not procedural)

Licensing and Deployment models …

© attachmate 2004

So…

Today, the value is not defined as much by functionality anymore but by connectivity However, we need a new programming model

Why SOA today? We are reaching a new threshold of

connectivity and computing power

Mainframe Client Server Web SOA

Constructing Software In a Connected World

From Components to Services

© attachmate 2004

Constructing software in the web era (J2EE, .Net, …)

App ServerApp Server

Client Client

DB

CCI CCI CCI

ERP CRM

requestresponse

Model

ERPEAIEAI

b2b

Internet

CCI: Client Communication Interface

Controller

View

© attachmate 2004

Why do we Want to Move to a New Application Model Today?

We are moving away precisely because of connectivity J2EE, for instance was designed to build 24x7 scalable

web-based applications Job well done

But this is very different from, “I now want my application to execute business logic in many other systems, often dynamically bound to me” JCA (J2EE Connector Architecture) is not enough EAI infrastructures are not enough

© attachmate 2004

A Component now Becomes a Service Running Outside the Consumer Boundaries

DB

CCI CCI CCI

ERP CRM

Service Service Service

Registry

11register

ConsumerConsumer

SOAP SOAP SOAP

XML XML XML

33 invoke22

Discover and/or Bind

Policies

© attachmate 2004

From Components to (Web) Services

Requires a client library

Client / Server Extendable Stateless

Fast Small to medium

granularity

Loose coupling via Message exchanges Policies

Peer-to-peer Composable Context independent

Some overhead Medium to coarse

granularity

© attachmate 2004

What Happens to the Technical Services Typically Provided by an Application Server?

Transaction Security Connection pooling Naming service Scalability and failover …

They become the “Service Fabric”

© attachmate 2004

What about the notion of “Container”? They become Service “Domains”

The notion of “container” shifts to the notion of “Domain Controller” A domain is a collection of web services which share

some commonalities like a “secure domain” A container is a domain with one web service Web Services do not always need a container

Consumers invoke the domain rather than the service directly

This concept does not exist in any specification…

© attachmate 2004

A Service Fabric can be more than a Bus with a series of Containers / Adapters

DB

CCI CCI CCI

ERP CRM

NEP NEP NEP

ConsumerConsumer

DomainController

ReliableMessaging

Process

RegistryTxXML XML XML Registry

register

Discover and/or Bind

Policies

© attachmate 2004

The road to SOA…

NEPNEP NEP

NEP

NEPNEP NEPNEP

Existing business logic, often “model-oriented”

Coordination logic (Tx, Process, …) Metadata driven

NEPNEP

NEPNEP

NEPNEP

NEPNEP

“Global” business logic (tax, trade, …) and data access

NEPNEP

NEPNEP NEPNEP NEPNEP

User Interface NEPs

© attachmate 2004

Shift To A Service-Oriented Architecture

Function oriented Build to last Prolonged development

cycles

FromFrom ToTo

Coordination oriented Build to change Incrementally built and

deployed

Application silos Tightly coupled Object oriented Known implementation

Enterprise solutions Loosely coupled Message oriented Abstraction

Source: Microsoft (Modified)

© attachmate 2004

So Migrating to SOA

Would entail Organizing the business logic in a context

independent way Typically, model oriented business logic is

wrapped behind (web) services

Re-implementing the controller with “coordination” technologies

…The view must be completely re-invented

© attachmate 2004

From this point on…

We will focus on 3 key aspects of the controller The coordination layer Information Entities The relationship between BPM and SOA

© attachmate 2004

The “Coordination” Layer

Many, many, many overlapping concepts not layered, not architected Composition Orchestration Choreography Collaboration …

What is the relationship between these concepts?

© attachmate 2004

Coordination Specification

OASIS/WS-CAF Context management Coordination Transaction management

As one possible, yet very important example of coordination

© attachmate 2004

What is Context?

S1S1

S2S2

S3S3CTX

ServiceCTX

Service

S4S4

Peer to peer interactions mandate a “context” service

(e.g. in this case S3 may need to know the state of interaction betweenS1 and S4 to provide its service)

© attachmate 2004

What is an activity Lifecycle Service?

S1S1

S2S2

S3S3CTX

ServiceCTX

Service

S4S4

ALSService

ALSService

ALS allow to demarcateunits of work shared across several services

© attachmate 2004

What is Coordination?

S1S1

S2S2

S3S3CTX

ServiceCTX

Service

S4S4

ALSService

ALSService

CoordinationService

CoordinationService

A coordinator is an active componentof the architecture

It can support a service or provide services itself

Multiple purposes:- Transaction- Orchestration- Choreography …

© attachmate 2004

What are the Coordination Topologies?

A BBinary relationship• Context and Activity are most often implicit• Self-coordination

Hub

A

B

C

D

E

F

Hub and Spoke relationship• Context and Activity are handled by the hub• Coordination is handled by the hub exclusively

Coordinator

CTX

A

B

C

D

E

F

ALS Multi party peer-to-peer relationship• Context and Activity are explicit• Context, ALS and Coordination are handled by the fabric

© attachmate 2004

Coordination is a key abstraction

Rely on generic fabric services for types of coordination Not everything is a process…

Different types of coordination can be composed A transaction may include an orchestration

definition as part of an activity An orchestration definition may contain

several transactions

© attachmate 2004

Information Entity in SOA

“at the heart of Web services is a very complex problem: with distributed applications comes the need for distributed data sharing” Identification and equivalence authentication Authorization and privacy mediation synchronization

Source: The Dataweb: An Introduction to XDI, Drummond Reed et al.

© attachmate 2004

Information Entities in SOA

Several dimensions appear when managing an Information Aggregate in a SOA

InformationEntity

InformationEntity

Identity

Content

State

Location(s)

Replication

Privacy

Specificto SOA

© attachmate 2004

Key problems to solve

Isolation We cannot be guaranteed that the information

we have is the information held by the system of record

Containment We cannot be guaranteed that a service

consumer is going to apply the same privacy rules to the information provided to it

© attachmate 2004

Web standards vs. Dataweb standards

URIs XRIs

HTML XML/XDI

HTTPXDI/HTTPXDI/SOAP

100% resourceaddressability

Common representationand linking format

Common interchangeprotocol

Web Dataweb

Source: Drummond Reed

© attachmate 2004

Websites vs. Dataweb sites

HTML

WebsiteA

HTML

HTML HTML

HTML HTML

HTML HTML

HTML HTML

WebsiteB

XDI Linkcontracts

XML/XDI

Dataweb siteA

XML/XDI

XML/XDI

XML/XDI

XML/XDI

XML/XDI

XML/XDI

XML/XDI

XML/XDI

XML/XDI

Dataweb siteB

Source: Drummond Reed

© attachmate 2004

Information Elements and Aggregate Represent a Big Challenge in SOA

We have barely scratched the surface Thinking that we can get away by saying that

we are simply exchanging messages between services is IMHO a mistake

SOA will not exist without its concept of information entity Entity beans were clearly not a good solution .NET offers the concept of DataSet which I like

better

© attachmate 2004

SOA and BPM

SOA is about constructing software components that can be reused in context unknown at design time Composition versus Extension (OO)

BPM is about being able to precisely model and possibly change the context in which enterprise components are used

But how the two meet?

© attachmate 2004

Entity

Elements of BPM

Activity

State

Event

Actor

Contentperforms

Initiates

changes

generates

ServicesRepresented by

© attachmate 2004

How is BPM perceived today in the SOA community? Two approaches

Event Oriented BPML, BPEL Pi-Calculus (Also Event Calculus)

Activity Oriented WfMC Petri nets

IMHO, the approaches must be combined and state must be part of the model

“Turing complete” is the excuse for remaining “pure” (e.g. event oriented only)

Source: Paul Brown, FiveSight,

© attachmate 2004

BPEL

“BPEL is an XML language for defining the composition of (web) services into (new) services” (Paul Brown, FiveSight)

BPEL is above all a simple Orchestration language not a Business Process Language

BPEL would require that every process Either has a “center” of execution A process is composed of a large set of orchestration

definitions interacting with each other In pi-calculus, even a variable is a process…

BPEL assumes that business processes can be fully captured in a single definition, including all possible exception paths Not sure this is the right assumption

© attachmate 2004

ERP

ManufacturingCapacity Analysis

Manufacturing Execution (MES)

Manufacturing/Production

Planning

Sync DispatchList

Get DispatchList

Show DispatchList

Sync ItemMaster

Sync ProductionOrder

Sync Routing

Sync BillOfMaterial

Sync ItemMaster

Sync ProductionOrder

Sync Routing

Sync BillOfMaterial

Syn

c Item

Syn

c Pro

dord

er

Syn

c Rou

ting

Syn

c BO

M

Co

nfirm

Invento

ryIssue

Up

date

WIP

Con

firm

Up

date

WIP

Con

firm

Co

nfirm

Invento

ryIssue

OAGIS 8.1 Scenario 50

A business process does not have a “center”…it is de facto “peer-to-peer”

© attachmate 2004

A very simple example

A buyer orders some goods from a supplier The supplier performs a series of steps to

fulfill the order Approve the order Update the order entry system Update the billing system …

© attachmate 2004

Orchestration languages are a critical piece of the puzzle

They allow us to implement complex services which involve: Long running (from a few hours to a few

months), And event driven business logic

They are not about modeling Business Processes by themselves Different orchestration (i.e. different services)

can run within the same business process And vice versa

© attachmate 2004

A Business Process can be Viewed as a Multi-Party Choreography (not orchestration) of Peer Services

User ActivityUser

Activity

Buyer Supplier SFA Salesperson

Start

ERP

MappingRouting QuoteRFQ RFQ RFQ

Order Order

Order

Invoice

Accounts

Account

SalesTax.comSalesTax.com CreditCheck.comCreditCheck.com

Orders

BillingInvoice

SalesOrder

Events

Activity

InformationEntity

© attachmate 2004

Services in a SOA are orchestrated (BPEL) ! This is one of the best model to re-use existing business logic

Quote ServiceOrchestration Definition

RFQ

Nack

Quote

SalesTax

<<send>>quoteupdateDB

Transition

Message flow

RFQ

Quote

Ok?No

sendNack

Order

<<invoke>>calculateSalesTax

<<invoke>>GetQuote

Ok?No

EntityState

Entity

<<receive>>RFQ

© attachmate 2004

A Choreography Provides a Model of the Event Flow Between Activities

Buyer Supplier(Self)

Order Entry

POAckPO

BTA1

OpA1

POAckPO

Manager

OpA2

Salesorder

Start

Wit1

POPO

Billing

Failure Success

[BusinessFailure] [Success]

© attachmate 2004

So …

Orchestration « … is an emerging [concept] that would give

programmers a way to formally describe processes underlying business applications so that they can be exposed and linked to processes in other applications »

Choreography Is a concept that specifies how these processes are

linked together across the enterprise Choreography can be « active » when mapping and

routing are necessary They are both a form of Coordination

© attachmate 2004

Bringing BPM and SOA together

The foundation is becoming sound with strong theoretical support A big piece still missing: “state” (IMHO) Shift from Orchestration to Business Process Definition

Once the foundation is in place we should see Domain Specific Languages (DSL) appear that are a lot closer to the way the users (not the programmers) think about a Business Process

© attachmate 2004

A Technical Model for Constructing Software in SOA

We need to adapt the foundation of modern application architecture Model-View-Controller

Model Services Controller Coordination View ?

© attachmate 2004

The Model-View-Controller pattern Revisited

ViewView

ModelModel

QueryUI Controller

TaskEngine

BusinessProcessController

Task Request

SelectTask

ServiceRequest

ChangeState

Controller

ModelChanged

Model Changed

SelectView

WS

WS

WS

WS

© attachmate 2004

SOA requires a Complete Separation of the Business Logic and UI

ViewView

ERPERP PLMPLM CRMCRM

UI Controller

TaskEngine

BusinessProcessController

ServiceRequest

WSQuery

Engine

WS

WS

WS

WS

WS

WS

© attachmate 2004

Registry

Context ALS

It is likely that BPM will be the main paradigm for the « Controller » in a SOA

ViewView

ERPERP PLMPLM CRMCRM

TaskEngine

WS

WS

WS

WS

WS

WS

IntegrationServer(Decoupling forB2B and EAI)W

S

Orc

he

stra

tion

En

gin

e

Orc

he

stra

tion

En

gin

e

Orc

he

stra

tion

En

gin

e

Business Process Execution

Conclusion

The Future Belongs to Whom Can Master Connectivity

© attachmate 2004

Service Orientation is a New Computing Paradigm Not as a new name for

API Component

A genuine set of concepts with which we can construct new kinds of software This is as significant if not more than Object

Orientation

In particular SO forces us to think about enabling the same piece of code to be leveraged by large numbers of consumers in unforeseen context

© attachmate 2004

SOA is still far ahead of us

We still need to shape what SOA means I have emphasized 3 concepts but there are a lot more

and a lot more not even uncovered today BPM and SOA will complement each other

We need lots of work to achieve and deliver the SOA value and go beyond “toy” applications of SOA At best we are capable of delivering web services

today

Standards work is both a curse and a blessing They open the road but blur the space and bring

confusion because of the lack of … coordination

© attachmate 2004

We can expect

A new “gold rush” to own and publish application “services” Mainframe and Client Server applications (ERP, CRM,

SCM…) will be impacted dramatically

Far richer and smarter software Could also become a nightmare if we look at all the

security breaches that occur today

However, some key enabling technologies are still missing … Standards “convergence” is now of primary importance