10 SOA commandments - Agile Business Consultants 10 SOA commandments v1 3.pdf · 10 SOA...

Click here to load reader

  • date post

    18-Jun-2018
  • Category

    Documents

  • view

    220
  • download

    3

Embed Size (px)

Transcript of 10 SOA commandments - Agile Business Consultants 10 SOA commandments v1 3.pdf · 10 SOA...

  • AGILE BUSINESS CONSULTANTS BV

    10 SOA commandments Getting the most out of Service Oriented

    Architectures

    Hans Wierenga

    23/3/2010

    Service Oriented Architecture (SOA) is an approach to organizing information processing. It is capable of lowering the costs of information systems interoperability, by greatly reducing the number of aspects of these systems that have to be brought into agreement with each other for them to work together. If it can be successfully applied on a large scale, the systems landscape will differ from that of today in much the same way as todays freight industry differs from that of the pre-container era. However, it is currently being used in ways which result in additional overheads but fail to deliver on these interoperability advantages. Paradigms which are appropriate to database era are being applied to SOA, resulting in counterproductive, often stupid and sometimes downright dangerous designs. These paradigms must be replaced by patterns of thought and behavior which ensure that SOA leads to simpler interfaces, functionally better IT solutions and more manageable projects. This can be achieved by adhering to ten basic commandments.

  • Ten SOA commandments

    Author: Hans Wierenga Page 2 of 24 23-02-2010

    Introduction

    The potential impact of SOA

    Service Oriented Architecture (SOA) is an approach to organizing information processing. The

    approach consists of describing all interactions in terms of services, in which a requestor asks an

    agent for something to be done, and the agent ensures that it gets done and delivers a response to

    the requestor. This way of thinking can be applied at a business level, in order to describe

    interactions between organisations, at a functional level, to describe how the activities of which

    business processes are comprised interact, and at the level of information systems, in order to

    describe how systems and parts of systems interact. At each of these levels the principle is the same:

    how the agent does what needs to be done is no concern of the requestor, not even whether it is

    done completely automatically, completely manually or something in between. Nor is the requestor

    concerned whether the agent delegates part or even all of the work to others. All the requestor

    needs to agree on with this agent is how the request and the response should be formulated, and

    what the effect of the service is.

    SOA is widely touted as a paradigm with enormous potential to reduce the costs of systems

    development, testing and maintenance. In particular, it promises to lower the costs of getting

    information systems components to work together, by greatly reducing the number of factors that

    have to be brought into agreement with each other. Using SOA, differences in things such as

    computing platforms and data formats are much less of a barrier to communication between

    systems than with earlier paradigms. That makes cooperation on a larger scale possible, because it

    minimises the hindrances imposed by the need to get systems designers to agree with each other

    and, for that matter, to get systems configurators to agree with each other. If the promise can be

    realised, the consequences will be revolutionary. Like the car changed the shape of urban regions,

    the shipping container revolutionised the freight industry, and the lowering of transaction costs

    enabled modern free market economies to develop, SOA will enable new patterns of cooperation.

    When SOA dominates the way we apply IT, the systems landscape shall differ from that of today, as

    a city designed around cars differs from a city designed around trains. For those of us whose thinking

    is dominated by current technologies, it is difficult to imagine how big a difference SOA can make.

    But the flexibility advantage that SOA offers is like the advantage of cars over trains: whilst trains can

    be made to go as fast as cars, they can never support door to door transportation in the ways cars

    can. It is simply not practical to put a train station at the end of every driveway, or even rails in every

    street.

    Why this impact has not (yet) materialised

    To reap the benefits of the new paradigm, we must make use of the new possibilities it offers.

    Unfortunately, most of the current hype around SOA has little concern for these possibilities. The

    major discussion seems to be concerned with establishing how SOA enables individual information

    systems to be developed more rapidly. However, that is not where SOA adds most value. In fact, its

    debatable whether SOA is really an improvement on previous approaches in which separate

    functions cooperate by sharing a common pool of data, normally implemented in a database. Using

    SOA to build an individual, isolated information system is rather like using shipping containers to

  • Ten SOA commandments

    Author: Hans Wierenga Page 3 of 24 23-02-2010

    move goods around your manufacturing plant: certainly, it imposes order and structure on your

    internal logistics, but the containers get in the way more than they help. SOA results in better

    interoperability between information systems, rather like shipping containers enable interoperability

    between carriers. Thats an important advantage, because the lead time between requirements

    finalisation and the information system becoming operational is often largely determined by the

    interoperability. Getting an information system to work together with the systems in its operational

    environment normally costs more time and energy than building the system as such.

    The focus on the use of SOA within, rather between information systems is a symptom of a wider

    problem: because SOA is seen as a new way of doing what we have always done, we are unable to

    reap its benefits. SOA concepts and technologies are being used with current systems development

    paradigms. These paradigms have been developed in the database era, and incorporate the

    limitations of database technology. Applying SOA with these limitations will result in additional

    overheads, but fail to deliver on the additional benefits. But these databased paradigms are so

    pervasive and pernicious that we are often not aware of how much they affect our thinking. They

    are so ingrained that we regard them to be just plain common sense. Unfortunately, they are

    generally counterproductive, often stupid and sometimes downright dangerous. They lead to

    solutions that combine the disadvantages of the databased era with the down side of SOA, whilst

    failing to deliver on the advantages that SOA has to offer.

    What needs to change

    The SOA paradigm has its own common sense rules, and these are often completely different to

    those of databased paradigms. There are ten basic ones. The first five are concerned with making

    things simpler than the databased paradigms dictate simpler in the sense of sticking to the

    essentials. When we make things simpler in this way, there is a dramatic improvement in the

    probability that different solutions to the same problem will be interoperable. The following four are

    concerned with making IT solutions better than their databased equivalents, by avoiding the blinkers

    that lead databased thinkers to develop ineffective solutions. The last is concerned with making IT

    more manageable, and in particular with organising systems development so as to reduce project

    complexity and risks. SOA makes this possible but also necessary because it allows more

    functionality to be delivered as infrastructure.

    Making it simpler

    The rationale

    In a trapeze act, effective cooperation is based on each trapeze artist being in complete agreement

    as to where the other is going to be at each point in time. Some trapeze artists are so confident of

    knowing this that they regularly perform blindfolded. They can find each other because they have

    ensured that at any given time they can each have only one possible location.

    The successful application of SOA in order to maximise interoperability is very much like a trapeze

    act. Interoperability demands that the millions of possible solutions as to how parties communicate

    have been reduced to just one, on which both parties can rely. That doesnt have to mean that there

    is something wrong with all the other solutions: whether we drive on the left or on the right is

    arbitrary, but it is important that we all drive on the same side of the road.

  • Ten SOA commandments

    Author: Hans Wierenga Page 4 of 24 23-02-2010

    When just one party performs a service for just one other party, it makes little difference as to which

    solution the parties agree, as long as they agree. Idiosyncrasies from one side or the other can

    determine the solution, without making the total communication effort significantly greater. After

    all, these idiosyncrasies will need to be handled somehow, whatever the solution. But if there are

    many parties requesting the service or many parties performing the service, the picture changes.

    Then it is important that the communication solution is reduced to the essentials, in such a way that

    each party has to handle its own idiosyncrasies and no other party is confronted with them.

    It is illustrative to compare information communication with transplantation surgery. For a

    successful transplantation of an organ from one person to another, the transplanted organ must

    match the recipient on a wide range of factors, the vast majority of which have nothing to do with

    the biological function of the organ. In other words, the transplanted organ must have the same

    idiosyncrasies as the recipient. As a result, our organs cannot be transplanted as if they were Lego

    blocks. Currently, information communication is very similar. For one information system to perform

    a service for the other, they must agree on a wide range of things. They must use the same

    vocabulary (metadata), the same set of functions which can be invoked by the one and executed by

    the other, the same expectations for the data content of the questions and answers for each of

    these functions, the same coding systems, the same technical communications protocols, the same

    addressing scheme in order to channel messages to and fro, compatible notions of the tempo in

    which a reaction is expected, compatible techniques for ensuring that messages dont get lost,

    compatible authentication mechanisms in order to ensure that they are talking to each other rather

    than to imposters, compatible encryption techniques and key management in order to ensure that

    messages cannot be listened to or intercepted, and so on. In order to facilitate interoperability, we

    must ensure that parties that start standardising on these things independently of each other arrive

    at compatible standards. This is possible only when some very rigorous rules are followed, so that

    interfaces are reduced to the essentials. This way there is no room left over for idiosyncrasies.

    The rigorous rules are all concerned with making service interfaces simpler. The less we need to

    specify, the smaller the room for disagreement.

    1. Dont know what you dont need to

    What you dont need to know cant hurt you

    The essence of the SOA paradigm is that it enables maximal cooperation with a minimum of

    agreement between the cooperating parties or systems. That is an enormous advantage, because

    anything you dont need to know doesnt need to be tested or maintained either. What you dont

    need to know cant hurt you. Given that 40 percent of the systems development costs go into testing

    and up to 80 percent of the lifecycle costs of an information system are incurred in the maintenance

    phase, anything that the SOA paradigm liberates you from having to know represents money you

    can save.

    Metadata

    The most important thing you dont need to know is the structure, meaning and allowed values

    the metadata of any data which is not used by logic that is built into your system for selection,

    sequencing or performing calculations. You dont need to know this, because SOA technologies

  • Ten SOA commandments

    Author: Hans Wierenga Page 5 of 24 23-02-2010

    make the metadata available wherever the data is. Your system can interpret this metadata on the

    fly, so that you dont need to build the knowledge of that metadata into your system if all you want

    to do is capture, present or pass on the corresponding data. With a sufficiently sophisticated

    presentation facility it is even possible for users to perform all sorts of ad hoc selections and

    calculations using only the metadata that has been provided together with the data, rather than

    inbuilt metadata.

    By interpreting the metadata that is provided along with the data, rather than building the metadata

    into the system, your system does not need to be changed when the metadata changes. Only the

    source system needs to be changed. Think of the savings in development, testing and maintenance if

    you can apply this principle! Remember that making a change in two systems rather than one is, on

    average, four times as complex, because of all the coordination involved.

    For many consumer-facing systems, presentation and ad hoc selection covers most of their

    functionality. Such systems require inbuilt metadata for only the most basic customer data. That

    does not include current and past orders, customer contacts, photos, correspondence and whatever

    other data may be available for display, all of which can be presented in a manner which does not

    require any knowledge as to the nature of this data to be built in to the system.

    Technology

    Many of the things you dont need to know are technology-related. With SOA, you dont need to

    know whether the system with which you are interfacing is implemented using a Software-as-a

    service or not, where the computer on which it is implemented is located, what type of computer it

    is or which operating system it is running on, how the firewalls are configured, which database

    management system is being used, or which transaction management system is available. Other

    things that you dont need to know concern the internals of the systems with which yours

    communicates. In particular, you dont need to know anything about the metadata which it uses for

    its internal data storage, because any transformations that the other system may require in order to

    conform to the XSD are its problem, not yours.

    Having said that, there remains a wide choice of technology-related standards on which parties that

    communicate using SOA must come to agreement. In particular, there are so many that standards

    concerning web services that SOA practitioners refer to them collectively as WS-* hell (with the *

    indicating that there are many possible labels that can replace it). In part, these standards have

    arisen precisely because the SOA community has not been content to not know what it didnt need

    to know; this white paper presents guidelines to reduce the impact of the problems that gave rise to

    such standards. When these guidelines are followed, SOA requires substantially less prior agreement

    than other approaches.

    Designing stable interfaces

    If you want to reap the benefits of SOA, not knowing what you dont need to know must become

    your second nature. Apply it everywhere! For example, when designing a service to order goods,

    remember that the service requestor needs to know whether the goods will be available when he

    wants them, but does not need to know the current stock level. And if your application invokes a

    security service in order to determine whether a requested action is authorised, dont build in any

    more knowledge of the workings of the service into your system than is necessary. For example, if

  • Ten SOA commandments

    Author: Hans Wierenga Page 6 of 24 23-02-2010

    the security service uses the credentials supplied with the input to the application, all it has to do is

    pass them on! As far as you are concerned, they are just a single data item which is encapsulated in

    your incoming message. Dont even check whether the credentials are well-formed XML. If, for

    reasons known only to them, the security boys have chosen to deviate from standard SOA practice

    or to encrypt the credentials, that is their problem, not yours. If they change anything to the

    credentials, your application should not need to be changed or adjusted in any way. Anything you

    dont need to know cant hurt you. Unless, of course, you are so stubborn as to want to know it

    anyway, in which case perhaps everybody is better off if you dont waste your time on SOA.

    Not knowing what you dont need to know is harder than you might think. If you develop

    information retrieval services specifically for a particular information system with which yours

    communicates, you are already on the wrong track, because you have incorporated some knowledge

    of that other system into yours. Any change in those requirements will necessitate a change in both

    systems. In general, a better approach is to expose your systems data using a limited set of retrieval

    services, which, when used in combination, cover all the information retrieval requirements of

    attached systems. For example, a product database may be exposed by separate services: a simple

    one with just the code, description, units and list price of the product, a service which exposes all the

    financial aspects of the product, and a service which exposes all the logistical aspects of the product.

    Many systems will be satisfied by the simple service, and most will need some but not all of the data

    of either the financial or the logistical service, and some will need to use both together, but there

    will be no system that needs a special interface on top of these. This way of working is known as the

    McDonalds approach: the customer assembles his own meal from standard products. Supporting

    this approach is no hardship, because you need such a set of services anyway in order to support

    customer-facing applications. You can even support very specific information requirements in this

    way, because data that is not needed can be filtered out before consumption. If you dont like the

    gherkin in your big Mac, remove it! The idea is that providing too much information is much less of a

    problem than providing too little, for the recipients systems can easily be programmed to ignore

    information he doesnt need, but are stuck if anything is missing.

    Not knowing what you dont need to know also results in profound simplifications of the information

    exchanges which are required in order to support business processes. In the SOA paradigm, when

    you ask another agent to do something for you, that is all that you do. You dont provide the agent

    with additional information which may be useful or even necessary for the agent to carry out the

    task. You ask for the dish, and it is the task of the cook to ensure that the ingredients which go into it

    are available. You ask what you want, and shut up. The agent may in turn request some or all of the

    information from you using an information retrieval service, but its his problem to decide what

    information to retrieve, when he wants to retrieve it and where he wants to retrieve it from. You

    dont need to know about it, and still less do you need to incorporate that knowledge into your

    system. This way, changes on his side will seldom require a change on yours. For example, nothing

    needs to change on your side if he decides to discontinue maintaining his own copy of your data.

    It is of course true that not knowing what you dont need to know results in inefficiencies. Some

    operations that can be carried out in a single exchange will now result in multiple steps. The

    McDonalds approach often results in several services being required where one would have been

    sufficient, and leaving the other side to retrieve the information that it requires almost always

    results in more messages than strictly necessary. There will be situations in which it makes good

  • Ten SOA commandments

    Author: Hans Wierenga Page 7 of 24 23-02-2010

    business sense to optimise these communication patterns. There will also be many occasions in

    which you will want to optimise the user interface, if only because current presentation facilities are

    not very good at providing users with attractive interfaces. But before you optimise, consider what

    flexibility you are losing. And never, ever optimise anything for which the functional requirements

    have yet to be stabilised.

    2. Dont know what you cant know yet

    Premature specifications freezes

    One of the real bugbears of databased paradigms is that it requires you to fix data structures in

    concrete before you know enough to be able to do that properly. Thats because they ignore a fact

    of life: users dont know what they want until they see what they dont want.

    It works this way: once you have finalised a data structure design, any subsequent changes are going

    to require messy database conversions, in addition to changes in each system which accesses the

    database. All of these changes must be coordinated, which is hard enough if all use of the database

    is limited to a single system, but gets much harder when there are many systems, particularly if

    some of them are managed by parties not under your control. In fact, making these changes is

    already a problem in the systems development phase. In consequence, database designs are frozen

    early on in the systems development phase, and data analysts bend over backwards to get the

    design right.

    The problem here is that data analysts have an impossible job. They have to finalise a design before

    the users can understand it, let alone appreciate how it will work out in practice. It is only long

    afterwards - once the system has been built - that users can get a feel for the system and assess how

    well it meets their needs. If there are any major problems with the data structure, it is far too late to

    fix them.

    SOA prototyping

    How is SOA different? , you might ask. When all is said and done, doesnt SOA need to structure

    data just as much as databased paradigms? The short answer is that SOA works just as well when

    requested data is processed by a human agent as when it is handled by an automated system, and

    people can interpret data even when it is not optimally structured. For example, users can

    determine whether a letter is being sent to another country even when its address is presented as

    line 1, line 2, line 3 and line 4, whereas an information system would need at least the country to be

    an identifiable part of the data structure.

    The long answer involves a discussion of SOA prototyping, which is carried out as follows:

    1. Identify the metadata which should be built in to the system. Put it in a primary namespace,

    and store its data in a DBMS according to its structure in the conventional manner. For

    example, for a customer this metadata may consist of the customer-ID and the customer

    name. Because this metadata is built into the system, it is possible to build logic into the

    system in order to use these fields, for example to retrieve records using the customer-ID

    and to sort and select records on the basis of the customer name.

  • Ten SOA commandments

    Author: Hans Wierenga Page 8 of 24 23-02-2010

    2. For each database record, put all the data that wasnt included in the primary namespace

    into a single XML-string which is appended to the database record as a single field. For each

    such XML-string, make a secondary namespace, develop an XSD, and add a single data item

    for the string to the primary namespace. For the initial implementation of a customer

    record, this string may include metadata for address lines 1, 2, 3 and 4. The XSD of the

    customer record itself will then contain metadata for three fields: the customer-ID, the

    customer name and the XML-string containing the additional customer data. The XSD of the

    additional customer data will include metadata for each of the address lines.

    3. Capture and present all data using XSD-based logic for each relevant hierarchy of database

    records, augmented by a single, standard validation service per XML string (including the

    string for the primary record). If at all possible, use a tool to generate interfaces using the

    XSDs, rather than programming it yourself. This tool must unpack the secondary XML-

    strings using the metadata that they contain, and must capture new records using both the

    primary and the secondary XSDs. The result is a working prototype which the user can use

    to assess the usefulness of the intended system.

    4. Adapt and change the XSDs and the validation services as necessary until the users are

    happy with the system. In the case of the customer record, a new XSD for the string may

    contain metadata for street address, zip-code, state and country. Data stored using the old

    XSD will still display correctly, so there is no immediate need to convert it. New records will

    be captured using the new XSD.

    5. Consider migrating data to conventional database structures and to the primary namespace

    once both the users and you are convinced that the metadata is stable.

    Of course, SOA prototyping is not always necessary. Apply it when the requirements are subject to

    so much uncertainty that it is more economical to prototype first and stabilise later. But remember

    that it is much easier to do than databased prototyping. In the databased era, prototyping was an

    option, but in the SOA era, its a way of life. Dont fix designs in concrete until you know they are

    right; its as simple as that.

    3. Dont do more than asked

    Product-driven service decomposition

    One of the main strengths of SOA is that it is a business concept which is translated into technology,

    and not, as is the case for the databased world, a technological concept trying to catch up with the

    business. In SOA, each service must clearly add value, not just in some abstract sense, but specifically

    to those who invoke it. Everything that happens does so because a service requestor asked for it to

    happen. By not implementing anything within a service that the service requestor did not explicitly

    ask for, services can be limited to their essential functionality. If we all do that, the result is an

    enormous increase in interoperability.

    The highest level services in SOA are the business transactions: a customer places an order for goods

    or services, which unless the order is rejected by the vendor results in their delivery. In the SOA

    paradigm, everything that happens between order acceptance and order delivery can and should be

    implemented as services too. For each intermediate product or state that is required in order to

  • Ten SOA commandments

    Author: Hans Wierenga Page 9 of 24 23-02-2010

    deliver on the order, the vendor requests somebody or some organisational unit not necessarily

    within the vendors own organisation to deliver it. Each of these intermediate services in turn may

    themselves be decomposed into services, and so on, right down to the level of elementary units of

    value addition.

    The hierarchical service decomposition is an essential part of the SOA paradigm, because it enables

    the delivery of services to be managed by means of service level agreements. It ensures that there is

    somebody responsible for each service, and that service consumers know what they are getting.

    Common intermediate products and common services

    Note that the service decomposition is all about products. Processes only come into it when the

    subservices of which a service is composed cannot just be carried out all at once. For example, if the

    service determineCustomerOrderAcceptability requires that the subservice determineCustomer-

    Creditstatus is carried out after the subservice determineOrderValue, then the implemention of the

    service involves a process. Each such process is concerned with only one service. If you find you have

    a process which cannot be limited to a single service in this way, you have probably forgotten to

    model the initial customer request as a service.

    It is possible that the same intermediate product and hence the same service may be required

    for different higher level products. For example, business products that require substantially

    different processes to be delivered will normally differ hardly at all when it comes to the

    intermediate product in which the customer pays in order to extract money from the customer we

    need an amount, a date and a statement of what gives us title to ask for the money, regardless of

    what that was. In such cases, the normal line is that if there is a common intermediate product, it

    should be implemented by a single service that can be invoked from multiple higher level services.

    Only the result of such a service matters not the starting point because collecting the information

    that is necessary to deliver the service is part of the service, not of the environment from which it is

    requested. Note that the decision whether to have a single service is a business decision; it has

    nothing whatsoever to do with information technology. If it is practical and useful to deliver the

    same intermediate product both now and in the relevant future, there should be a single service.

    But if there are business forces at work which now or in the relevant future will result in structurally

    different intermediate products, we are better off with separate services for each set of

    requirements. The bias is, however, towards single services. In SOA, things are the same until they

    are different, whereas in databased approaches, things are different until they are the same.

    Common services with multiple behaviours

    What, you may ask, is the point of making something one service if it requires two drastically

    different types of behaviour? Arent we busy with the same One size fits all restrictiveness that has

    bedevilled the databased world? For example, if we sell two types of product, one of which employs

    fixed prices whilst the other calculates them afresh according to some complicated formula, why not

    have two billing services, each tailor-made for specific types of product?

    These are the right questions, asked at the wrong place. They are the right questions, because any

    design approach which cannot adequately handle the variation that occurs within the problem

    domain is doomed to failure. But they are asked at the wrong place, because the flexibility to adapt

  • Ten SOA commandments

    Author: Hans Wierenga Page 10 of 24 23-02-2010

    solutions according to the variation in the problem domain should be built into the services, rather

    than be built around them. In the case of the billing service, it should determine which of the two

    billing algorithms should be employed, each time it is invoked. That way, if a third billing algorithm

    were to be introduced, only the billing service needs to be modified. Note that this does not mean

    that the billing service has to be designed as some sort of all purpose machine; it could equally

    well invoke separate subservices for each algorithm. This choice the choice between a

    multipurpose solution and a framework into which different components can be slotted is a choice

    that can be made per service, because it does not need to be known by the service requestor. In

    SOA, this choice will more often than is currently the case result in framework solutions, because the

    perceived need for multipurpose solutions is in large part the result of the failure to separate service

    requests from the information which is needed to carry them out. The days of one-size-fits-all

    multipurpose solutions with so many parameters that it requires highly qualified specialists to

    implement them are numbered.

    Sticking to the essentials

    The process driven approach also enables us to distinguish between the things we do on behalf of

    the service requestor and the things we do on our own account. The things we do on behalf of the

    service requestor should be carried out as part of the service, and the rest should not. By making this

    distinction, we can keep the service as simple as possible, in order to be able to replace it later

    without having to change all sorts of other things as well. For example, we may manufacture

    products in order to satisfy an external service request, but maintaining the bookkeeping system is

    something we do for our own needs, not those of the requestor. If we were to develop a service in

    which a customer order is translated into both the manufacturing activities and the bookkeeping, we

    would need to amend the service whenever we implemented a new bookkeeping system. That may

    not sound so bad, but when we consider all the things we do on our own account it is. Bookkeeping,

    logging, stocking data warehouses with up-to-date data, maintaining employee productivity data: all

    of these things and more are typically carried out using systems that differ from one organisation to

    the next and from one time to the next, so that incorporating knowledge of them within a business

    service reduces interoperability and increases the difficulty of replacing the service with an

    implementation via another system. We can avoid this problem by generating notifications

    whenever something that is important to these functions occurs, and letting them retrieve the

    information they need to process the event using common services.

    Another category of business activities that should not be carried out as part of a service consists of

    those activities which you will not reverse if the service request itself were to be retracted. Typically,

    this includes activities such as ordering supplies because the customer order resulted in stock levels

    under the replenishment level, registering a new customer and updating the information of an

    existing customer. These activities form separate steps in the process, which should be executed

    using separate services.

    Of course, this way of thinking can be bolted on to databased paradigms. But it is not inherent to

    them, as it is with SOA. In consequence, many SOA implementations betray the databased thinking

    which inspired them by identifying services in a bottom-up fashion, rather than the SOA top-down

    way. In a bottom-up approach, a service that was initially developed for one problem can be re-used

    for another problem as well, thanks to the fact that the person who designed it gave serious thought

    as to how to make it more generally applicable. In SOA, the use of a single service in multiple

  • Ten SOA commandments

    Author: Hans Wierenga Page 11 of 24 23-02-2010

    contexts is the result of deliberate design, not of intuition, and it is designed for all of those contexts

    from day one. To talk about re-using a service which delivers a common intermediate product is like

    saying that you re-use your front door if you use it to you enter your house regardless of whether it

    is to access the living room, the kitchen or the bathroom. Can you imagine a kitchen designer saying:

    Isnt it wonderful? The guys who designed the exterior entry to the living room just happened to

    make something which I can use to provide exterior entry to the kitchen!? Anybody who talks

    about re-use of, say, a billing service just hasnt got it. Mind you, its not wrong, just strange and

    not very illuminating.

    Services follow the business

    One consequence of the SOA way of thinking is that SOA results in services that are expressed in

    terms of their propositional meaning, not their mechanical implementation. For example, a service

    to add a customer record is databased, whereas a service to register a new customer is SOA, even

    when they do exactly the same thing. The label we give to a service is important because it tells us

    something about who is asking for this to happen and why he wants it. In this particular case, the

    SOA top down approach leads to the conclusion that a business process needs a valid customer

    registration, which should be accomplished by amending an existing registration if there is one,

    rather than automatically creating a new one. In SOA, this is the responsibility of the customer

    registration service, whereas a databased approach pushes this responsibility on to the service

    requestor. Similarly, a SOA registerCustomer service will itself decide what the customer-ID is going

    to be, whereas a databased service may just as well allow the service requestor to decide this.

    Convergence to a single solution

    In general, SOA top-down thinking results in many design decisions having only one valid option,

    whilst a databased thinker will see that option as just one of the possible choices. Thats an

    important SOA advantage, given that SOA is oriented toward interoperability, and interoperability

    demands that we all drive on the same side of the road rather than negotiate this with each car we

    meet. Those who ignore this for example by maintaining that web services are just one of the

    many ways to implement SOA across systems boundaries have about as much chance of durable

    success as a chess player who thinks only one move ahead. True, there will often be simpler ways

    than web services to connect system A to system B, but what are you going to do in order to make

    the same data available to the call centre or to an output management facility? What are you going

    to do in order to ensure that the data warehouse is informed of the events which you transmit from

    one system to the other, or to inform your customers of such events as soon as they happen? In the

    SOA world, data and events must be made available to many systems, and web services are the most

    effective way of ensuring that this can be done with low investments and low maintenance costs.

    One domain in which this difference is evident is EDI. Classical EDI is aimed at determining which

    information may need to be communicated between organisations, and defines a vocabulary for

    that information. That is not the same as defining a particular information exchange. For example,

    you can use the same EDI-message to place an order, to query its progress and to amend it. Two

    organisations who want to use these specifications to communicate must sit down and agree exactly

    how they are going to use this vocabulary. SOA separates these concerns: vocabulary is handled

    using namespaces, and these namespaces may be used in multiple services, each of which has a

    specific purpose.

  • Ten SOA commandments

    Author: Hans Wierenga Page 12 of 24 23-02-2010

    Another domain in which the SOA top-down approach leads to more specific conclusions is the

    question as to what constitutes the boundaries of an orchestrated process. Typically, this is a

    question which leads to endless debates. For example, is the delivery of a confirmation message to

    the customer using an output management service something that should be orchestrated as part of

    a customer process, and if so, should it be a fire-and-forget affair or should the output management

    service report the successful completion of the action? In SOA terms, all of these things are part of

    the purpose of the customer, and should therefore be orchestrated. Other actions, such as updating

    the data warehouse to take account of the new customer order or updating the general ledger, are

    clearly not part of that purpose and should not be included in the orchestration of the process, even

    if they are carried out by the same party.

    4. Dont do your own chores

    Generic functions

    A business service should have inbuilt logic only for those functions which are specific to the service.

    It should delegate all other functions. That way, the service can itself be kept as simple as possible.

    That makes it more easy to design and test, and, if need be, to replace. It is a matter of elementary

    mathematics: the greater the number of factors on which a new software components must match,

    the less likely it is that there is some common of the shelf software (COTS) that fits the bill, and the

    more expensive such a solution is likely to be if it happens to be available. If you can delegate the

    non-specific functions using a standard service, that reduces the number of factors on which a match

    is necessary.

    The primary tasks which the service can delegate are chores: tasks which are concerned with the

    housekeeping rather than the actual business functions. These chores are generic by nature, in other

    words the way in which they are done is not specific to the business context of the services which

    they support.

    Generic user interfaces

    It may surprise you to learn that the biggest chore that an information system should not do itself is

    managing the user experience. This is generic functionality, and should be handled as much as

    possible using a standard tool. The user experience includes the work lists from which he may select

    work items to carry out, the facilitation of work item execution for example by launching custom

    user interfaces, if we still have any and signing off completed work items. It includes selection of

    transactions which the user may perform, the presentation of input screens normally generated

    from XSDs and their validation using the standard validation services corresponding to the

    transaction. It includes maintaining the current user context so that he doesnt need to re-enter the

    current customer, product, case, process instance or whatever but can use these values by default

    and override them when necessary. It includes the presentation of all documents which are relevant

    in the current user context. It includes the presentation of alerts on which the user may need to

    react.

    All of this can be implemented using a tool that requires no inbuilt knowledge of the business. It is in

    general much better for the user to have a uniform environment in which all of this is covered than

    to optimise a user interface for a specific type of action. Where there is a clear business case to

  • Ten SOA commandments

    Author: Hans Wierenga Page 13 of 24 23-02-2010

    deviate from this principle, remember at least this: dont optimise the user interface before you

    have stabilised it.

    Typical generic functions

    Other chores include, but are not limited to, the following:

    Security: Chores that are concerned with establishing the identity and access rights of

    service requestors.

    Notification: Chores that are concerned with identifying which recipients should be informed

    of a business event. This includes maintaining the subscriptions on which this is based.

    Output management: Chores that are concerned with communicating information offline,

    rather than as a service response. This is typically the case when a customer request must be

    formally confirmed, for example as an email that confirms the web purchase that you just

    made via your browser. It is also necessary for unsolicited messages, such as monthly bills.

    Output management must determine via which channel the information must be sent, and

    which address should be used. It should transform the message into a format which the

    recipient can take in, send the message, and add it to the document archive. Output

    management includes the maintenance of the templates which are used to transform data

    into user-intelligible messages and of the addresses and channel preferences of possible

    recipients.

    Data transformation: Chores that are concerned with translating data from one format to

    another, bundling separate services into one service in McDonalds terms, happy meals

    and de-bundling, slicing service requests into separate requests for different populations

    and de-slicing the responses, queuing and de-queuing, or protocol conversion.

    Process orchestration: Chores that are concerned with orchestrating a process, so as to

    ensure that the services of which it is comprised are executed in the appropriate sequence

    and only when relevant, that alerts are issued when deadlines threaten to be exceeded, and

    that interrupts caused by ancillary messages or elapsed deadlines result in new process

    paths being taken.

    Dossier management: Chores that are concerned with maintaining and accessing dossiers of

    related information. These dossiers may be virtual, in the sense that the information which

    is presented to the user when he asks for a dossier may be retrieved using queries. For

    content which is extracted from databases this is regarded as normal, but there is no

    particular reason not to apply the same approach to documents. In some cases this should

    be augmented with facilities to assign documents to dossiers specifically.

    Record management: Chores that are concerned with maintaining information that is not

    allowed to be changed.

    Bottom-up delineation

    The services that implement these chores do not form part of the business service hierarchy. They

    cannot be designed them in a top-down manner, because there is no top. For such services a

  • Ten SOA commandments

    Author: Hans Wierenga Page 14 of 24 23-02-2010

    bottom-up approach aimed at achieving maximum re-use is more appropriate. This allows the best

    service implementations to succeed and inferior ones to be removed from the stage.

    With databased approaches it is seldom practical to replace an inferior generic function with a

    better one, because that requires changes in all applications which make use of the function. With

    SOA, on the other hand, such a replacement is simple, provided that the rule for not knowing what

    you dont need to know has been applied, including the corollaries that a service request should

    contain no more information than is necessary to specify the request and that the service itself

    should take the initiative to ask for more information when necessary. Take, for example, an

    authorisation service which is called by an application in order to determine whether a particular

    user should be allowed to perform some function on the data of some customer for example: Is

    our employee Donald Jones entitled to access financial data regarding the Acme Widgets

    company?. A simple version of the service might be capable of handling only situations in which

    these questions can be answered using a table of employees versus functions. A slightly more

    complicated version might recognise that Donald Jones is a member of one or more groups and use

    the group authorisations in addition to the individual ones. Taking things a step further, the

    authorisation service may use the credentials of the user in order to differentiate between

    employees and customers, and allow the latter access to only their own data. A sophisticated

    version might use a standardised service call to the Business Process Management System or a case

    management system, asking whether Donald Jones has been assigned any responsibilities in our

    dealings with Acme Widgets. Preferably, the authorisation service will log the requests and the

    answers, but simple ones wont. The organisation can switch from one version of the service to

    another without any change to the applications which invoke it.

    It is also possible to design generic services to automatically configure themselves for the situation in

    which they must operate. An authorisation service could, for example, check whether there is a

    service that will tell it about the responsibilities that employees have been given for particular

    customers, and decide to use only individual and group access rights if such a service is not available.

    In this way services can be made re-usable across many organisations.

    5. Dont bore yourself with testing

    Why SOA is easier to test

    One of the perceived disadvantages of SOA is the supposed difficulty of testing. This perception is

    entirely inappropriate, for many reasons.

    Firstly, the use of metadata in SOA makes it possible to prevent errors from being incorporated into

    the system. It is possible to validate the system at the metadata level, for example to ensure that all

    data which is required for processing is collected and checked before it needs to be used. This can be

    done across the breadth of a business process. Dont test the system when you can validate the

    design instead.

    Secondly, almost all the testing, and all the systems integration testing, can be done automatically

    once a test base has been established. There are, however, some prerequisites to this. There must

    be a rigorous separation between presentation and processing. Fortunately, this is a natural way of

    working with SOA. For all input, there must be an XSD. Using this XSD, test records are made. In the

  • Ten SOA commandments

    Author: Hans Wierenga Page 15 of 24 23-02-2010

    same way, test records are made for all expected output. Where processes do not produce all the

    outputs which are needed to prove that the system is working correctly, the test script must include

    queries which produce additional output for this purpose. When the test is run, the input records

    are presented to the system one by one, and then the actual output is automatically compared with

    the expected output. This results in a list of exceptions, each of which must be accounted for. The

    test can be run as often as needed. Naturally, the results may depend on the data which has

    accumulated in the database, so this needs to be compensated for. Also, the system must not exhibit

    time-dependent behaviour. Rather than waiting, say, a week for a time-based trigger to go off, the

    system must be capable of reacting to events which can be set off at time intervals which are more

    appropriate to an automatic testing sequence. The user interfaces should be tested on a one-by-one

    basis, but should never be used in the integration tests.

    Thirdly, SOA designs tend to result in more robust systems: there is simply less that can go wrong.

    SOA reduces the number of factors on which information systems must agree in order to cooperate,

    so there is less scope for a design error which results in a disagreement on some essential factor.

    And if things do go wrong, they can be detected before they can do any damage. With SOA, before a

    message is processed, it is validated in order to determine whether it is well-formed and conforms

    to the relevant XSD.

    Realistic testing

    Lastly, the rigorous separation between test and production environments that is characteristic of

    the databased era is no longer necessary, and is sometimes not appropriate. This is possible because

    we are not really testing systems anymore, but the passage and processing of messages. SOA offers

    a triply safe, effective manner of distinguishing between test and production messages. Each

    message contains version numbers of the message itself and of each namespace it incorporates,

    excepting those which are encapsulated. And each message contains a tag which indicates whether

    it is intended as test or production. All that is required is a SOA gateway just inside the firewall which

    carries out the following processing on each incoming message:

    1. The message is checked in order to determine whether it corresponds to a known version of

    a known XSD (excepting the encapsulated messages).

    2. The message is checked to determine whether it is valid, using our copy of the relevant XSD.

    3. If the message is intended for production, we check whether the version numbers have been

    approved for production use. Only if this is the case is the message passed on to the

    production systems. All other production messages are rejected.

    4. If the message is intended for test, it may be passed on to the designated test versions of

    systems. In special cases, where the message is used only for data retrieval, this may be a

    production system.

    5. Only after a message has been adequately tested is the register of production versions and

    XSDs updated.

    This manner of working is not only triply safe, but also enables the routing of messages to be tested

    in realistic manner. It also greatly reduces the need for reconfiguration in order to make the

  • Ten SOA commandments

    Author: Hans Wierenga Page 16 of 24 23-02-2010

    transition from test to production. Because this reconfiguration is by its very nature untestable, it is

    a frequent source of error. Release managers compensate for this by releasing new software only in

    the middle of the night or at weekends, so that if something goes wrong the previous situation can

    be restored before anybody much has encountered a problem. But thats no way of working if the

    change will affect other organisations. SOA release management is so much easier!

    It is time that the prevailing wisdom about SOA testing is revised. SOA is a way to minimise the need

    for testing and the effort to set tests up. It enables crucial tests to be done more automatically, with

    better result.

    Make it better

    Rationale

    SOA enables the development and deployment of information systems that support a far richer user

    experience than is generally achieved using databased approaches. The systems encompass more

    forms of information, have a broader repertoire of behaviour, achieve a higher level of uniformity

    and consistency in their behaviour and are more reliable, as measured from the points of view both

    of the customers and of those people within the organisation who are concerned with compliance.

    But achieving these benefits requires us to say goodbye to databased practices.

    6. Always keep your promises

    Why databases dont always keep promises

    The act of accepting a service request is by definition a promise to the requestor that the service

    request will be executed. This execution is by definition a process, involving at least one but often

    very many steps.

    Databased thinking and processes dont sit well together. By their very nature, databases are islands.

    And islands promote parochial thinking: anything outside the island cant be that important. This is

    illustrated most vividly by the databased concept of a transaction: a unit of work, which takes the

    database from one consistent state to another. In some special cases this concept may be extended

    to multiple databases, using 2-phase commit technology, but that is the limit. It is entirely foreign to

    the databased view of the world that logical consistency might need to be maintained across a

    business process, rather than just a moment in time, and across all places where information is

    changed, including not just databases but also process management systems, messages and the

    human agents that send and receive them.

    The SOA transaction concept

    Whats foreign to the databased world sits naturally with SOA. A business transaction and

    therefore the process by means of which it is implemented is a service. To understand how well

    SOA supports the logical consistency requirements, it is important to understand what a business

    transaction entails. A business transaction consists of the following elements:

    The vendor provides the customer with information on which the customer bases his

    purchase decision. Typically, the information is concerned with the properties of the goods

  • Ten SOA commandments

    Author: Hans Wierenga Page 17 of 24 23-02-2010

    and services on offer, the terms under which they may be acquired including, of course,

    the prices and their availability. In legal terms, the vendor represents this information to

    be true.

    The customer places a purchase order with the vendor, based on the representations made

    by the vendor.

    The vendor verifies that the information on which the purchase decision was based is still

    applicable, and if so, commits to the order. If there has been a change perhaps because

    the goods or services are no longer available, perhaps because the prices have increased,

    perhaps because the specifications of the goods or services have changed then some

    processing may be required in order to decide what to do: commit to the order anyway,

    negotiate amendments to the order, or perhaps just cancel it.

    The vendor and the customer each perform their side of the purchase agreement.

    How SOA maintains consistency

    SOA maintains the logical consistency of a business transaction in a number of ways. Firstly, all

    communication which implies a change to a previous situation can be done using a protocol that

    guarantees secure message delivery. Whenever a commitment is made by either the vendor or the

    customer, the act of making the commitment involves such a change. In this way, it is not possible

    for customer and vendor to have different perceptions as to the current state of the transaction.

    Secondly, the logical consistency of databases and the record of process progress in the process

    management system can be maintained using a 2-phase commit protocol. Logical consistency across

    multiple databases is maintained by a sequence of such 2-phase commits. First database A is

    synchronised with the process management system, then this system synchronises with database B,

    and so on.

    Thirdly, any changes in the vendors reality between the time that representations were made to the

    customer and the placing of the order are dealt with using an optimistic concurrency control

    mechanism. This way of working is natural to SOA: the process of determining whether there has

    been a relevant change can be completely automated. And because SOA makes it feasible to access

    data at the moment it is used to make a decision, it is only rarely that the optimism is unfounded

    and manual intervention is necessary.

    Lastly, an abort of the business transaction itself for example that the customer withdraws the

    order, turns out not to be able to pay, or dies can be handled relatively easily using SOA. Because

    the documents which have been used or produced in the context of the business transaction are

    clearly identifiable, it is possible to determine which countervailing messages are required in order

    to correct the perceptions of customers and vendors. Because it is clear which database updates are

    directly related to the transaction, it is clear which changes in databases need to be rolled back using

    compensating transactions. Given that aborts occur relatively infrequently, it is normally not cost-

    effective to completely automate these actions, but your design must allow for them.

    Note that the scope of a business transaction is limited to that which is done directly in order to

    handle a service request. In SOA, the rule is to orchestrate your own processes and provide event

  • Ten SOA commandments

    Author: Hans Wierenga Page 18 of 24 23-02-2010

    notification to others. If the service is aborted, it may be necessary to notify them of the change.

    What they do with this information is entirely up to them.

    Note also that creating the content of a customers service request is, strictly speaking, something

    that precedes the process, rather than part of it. Its not really your task: in principle it should be

    enough to supply your customer with an XSD and a validation service, and let him decide whether he

    is going to key data in or generate it from his own information systems in some way. For consumers

    this is not yet! a feasible approach, but it remains valid that collecting data and processing it are

    two separate things, performed in separate environments using separate tools. For the same data

    collection there may be many implementations, depending on the wishes of the customers and the

    channels that the use to communicate with you, but there should only be one service to process it.

    7. Dont limit yourself to models

    Databases are models, SOA is more

    A model of a domain is a representation of it which is easier to work with than the domain itself. The

    vast majority of databases are models. They represent some administrative, physical or social

    domain in such a manner that questions concerning this domain can be answered by a query to the

    database, and the need to act in the domain is signalled from within the database. It is, for example,

    easier to look in a database to find a customers address than to follow him home and note the

    address of the house he enters. And it is easier to count records in a database than persons in a city

    or products in a warehouse. The functions that need to be supported are in principle sufficient to

    determine the data structure of the database. However, because it is not normally possible to

    specify all such functions in advance, we use data modelling techniques to analyse the objects of

    interest within the domain, including their relationships and the items of information which tell us

    what we may need to know about them. For example, if the database is concerned with customers,

    there will typically be a database record for each customer, containing the information concerning

    the customer that the users of the database are interested in knowing. Because the data model is

    based on more fundamental considerations than just some particular uses of the database, basing

    the data structure on this model results in a database that is easier to adapt to unforeseen

    requirements. The database is semantically structured; in other words, it is structured according to

    the meanings of the things it represents.

    Databased technology is very uncomfortable with any data structure that does not implement a data

    model. It simply cant do very much with descriptions, documents, photos and the like. SOA, on the

    other hand, is just as good with documents and dossiers as it is with data that is structured according

    to a data model. SOA allows for a single design which handles both semantically structured

    information and documents. However, because databased systems dominate our thought patterns,

    we think it only natural to maintain their limitations when applying SOA. But in the SOA world there

    is no particular reason to apply such restrictions, and every reason not to.

    The benefits of going beyond models

    The primary reason to combine semantically structured data with such things as hyperlinks,

    documents, photos and sound fragments is that it creates a richer user experience. For consumers,

    this is not an option but an imperative. For the knowledge workers you employ, it makes them more

  • Ten SOA commandments

    Author: Hans Wierenga Page 19 of 24 23-02-2010

    effective. It is only for the staff performing routine administration that it gets in the way. But the

    more you apply SOA, the less you will need them. Their task of keying in data from forms can be

    outsourced to anywhere: all you need to do is scan each form and send the image to an agent who

    uses essentially the same web form as the customers to key it in. To the extent that the other tasks

    of these employees are genuinely routine, they can be automated.

    A second major reason to not limit yourself to databased data is that a synthesis of databased and

    document-based data is by far the easiest way to maintain the records and audit trails which are

    necessary for compliancy purposes. Databases are by their very nature unsuited to such ends.

    Because a database is typically built to act as a model of some administrative reality, it should

    therefore be easy to change it when that administrative reality changes. Database management

    systems are designed to facilitate such changes. When they are used to maintain records which are

    not allowed to be changed bookkeeping entries, for example the designers have to build all sorts

    of defences into the system to prevent malicious manipulation of the records. Even then, no judge in

    his right mind would trust a database. The records and audit trails which are needed for compliance

    purposes must be entrusted to record management applications, and linked to the databased

    systems using SOA.

    With SOA, connecting semantically structured data to other forms of information representation is

    easy. This makes it ideal for content management and content presentation. A database can contain

    hyperlinks to other information. For example, we can store the URL of the input document that gave

    rise to a transaction with the database record of the transaction. When the transaction record is

    displayed, the link is presented along with it, and may be activated by the user via a separate service.

    It is no more difficult to make a service to access a document from a document archive than it is to

    make one to access data from a database. Unless, of course, your document archive is not SOA-

    capable, in which case one of your first actions must be to replace it.

    But it works the other way around too. Not only is the semantically structured data enriched with

    the links to records, but the records are enriched with links to the semantically structured data.

    When you store a record in a document archive, the semantically structured data that is associated

    with it will normally contain all the information which is necessary to index it. There is little to no

    need to index the records manually. Better yet, the index can be progressively enriched by updating

    it automatically, every time that the transaction is progressed in the database.

    Integrating databased and document-based data has the further advantage that it is easy to support

    multiple versions of data. That doesnt make sense in the database world, because a database is a

    model, and to be useful as a model it must give at least a best guess in answer to any question about

    the administrative reality of which it is a model. A document, however, is a statement made by a

    specific party at a specific time in a specific context, and it is quite possible and often useful to

    know that conflicting statements exist.

    Culture shock

    Note that databased thinkers are not the only people who will have trouble with the SOA unification

    of the worlds of documents and of databases. People whose thinking has been formed by the

    current generation of document and record management systems have an even bigger problem.

    Suddenly, archived documents are part of business processes, not things which come into existence

  • Ten SOA commandments

    Author: Hans Wierenga Page 20 of 24 23-02-2010

    after process completion. And indexing is not something done manually, once per document, but

    something automatic and ongoing. Nor is the index part of the archive anymore: an archive only

    contains documents, and the indexes are maintained separately. And it is therefore not necessary to

    limit yourself to one archive, or even to only your own archive. These changes are so profound that

    it is hard to imagine that anybody who was suited to the old world could also be suited for the new

    one.

    8. Dont use models as photos

    Material and formal history

    The biggest mistake of my career was to design a database in which all service requests could be

    supplied with a time stamp, so that if it was filled, the service could reply according to the data

    contents that the database had at the moment indicated, instead of the current contents. It was

    enormously expensive, slowed the database down to a crawl, and served no useful purpose. When

    users wanted to see how a particular result in the past had been calculated, they invariably looked in

    the document archives, because the archives always contained the result that had been

    communicated to the customer. And thats all they wanted to know.

    Note that it is not the use of databases to register past situations that is the problem. Databases are

    models of some part of the real world, and the state of that part of the real world at some period in

    the past its material history may very well be relevant to the purpose of the model. But a

    database takes itself too seriously if it wants to recreate its own state at some point in the past, in

    other words to register its formal history.

    How SOA supports formal history

    The SOA approach offers a far more effective solution to the problem of recreating formal history,

    not just of a single database but of all the information which an organisation uses and creates in its

    administrative processes. The SOA separation between the collection of the data required to update

    the organisations information and the actual execution of the update allows that data to be

    archived in a record management application. Once SOA has been applied, this can be done

    automatically, with almost no additional effort in order to incorporate a new data set. In this way, all

    incoming messages and all updates made by your own employees are captured.

    The situation for outgoing offline messages is, if anything, even simpler. The SOA aversion to mixing

    chores and added value for customers leads quite naturally to the use of an output management

    service to ensure that offline messages reach the intended recipients via an appropriate channel, at

    an address which they really use and in a form which they can adequately take in. The output

    management service can take care of recording the outgoing information automatically.

    There is, however, one problem. The ease with which organisations can disseminate information

    using SOA tends to result in a substantial increase in the number of service responses to with

    external parties. These service responses typically imply a commitment of the organisation to their

    propositional content. The organisation could get into legal problems if they are incorrect. The

    standard SOA means of handling this involves the following guidelines.

    Implement business transactions in conformance to the SOA transaction concept.

  • Ten SOA commandments

    Author: Hans Wierenga Page 21 of 24 23-02-2010

    Include disclaimers in the service level agreements for all information retrieval services.

    Dont access data until you need to use it. That way it will be as up-to-date as possible.

    Dont lie. If you always tell the truth, you will encounter fewer claims that your information

    was incorrect.

    If the use of disclaimers cannot prevent you from having to respond to claims that your

    information was incorrect, log each service request and response, using the standard

    organisation-wide service.

    This manner of working is much simpler to organise than the databased solutions, because it does

    not require additional logic per message type, but only a call to the logging service, in which the

    entire incoming or outgoing message is treated as a single data item.

    9. Dont trust systems

    SOA hygiene

    If your body applied the same rules of defence against external agents as most web sites, you would

    be dead in a day. We live in a hostile world, where just about the only way to avoid your information

    systems being attacked is to make them inaccessible. In this world, SOA is a marvellously hygienic

    concept. In principle, the only messages we have to let through our firewalls are robustly enveloped

    XML-messages. And they can be passed through the firewall in such a way that they are checked for

    well-formedness, XSD-conformance and the validity of their authorisations before any further

    processing. The potential for them to commit mayhem is very small. Buffer overflows can explode

    only in bomb-proof areas, and SQL injections are nothing more than curses in a language to which

    we are entirely deaf. Corrupt messages are recognised as such before they can do any harm.

    It is important to note that the strength of this approach is based on the authentication and

    verification of messages, not of systems. That is necessary, because encapsulated XML is often

    passed on by systems that bear no responsibility for their content, so that checking that a message

    has come from some authenticated system doesnt guarantee much. It is more robust, because a

    successful hack of one information system or computer does not open the gates to hacks of others.

    SOA security contrasts markedly with the systems-based security of the databased era. With

    systems-based security, it is the task of each system to avoid buffer overflows, for each message

    type and each data item. There are thousands of points where it could go wrong. The chances of a

    weak spot are therefore considerable. A SOA gatekeeper, on the other hand, does all of this at a

    single point. Systems-based access security tends to result in authorisation controls being built in to

    systems, which not only makes it almost impossible to keep track of them, but also enormously

    expensive to introduce structural changes, such as the addition of users who are external to the

    organisation.

    A further advantage of using a SOA gatekeeper is that it can ensure that all outgoing messages are

    appropriately encrypted and signed, so that nobody can eavesdrop on them or change them en-

    route. In principle, most of the WS-*hell can be handled entirely by the SOA gatekeeper.

  • Ten SOA commandments

    Author: Hans Wierenga Page 22 of 24 23-02-2010

    Beyond systems-based authorisations

    Systems-based security also promotes mindsets which are much more determined by systems-based

    thinking than most of us would care to acknowledge. Because systems are structured around

    functions and data structures, authorisations are expressed in these terms. For example, an

    employee is either authorised to change product prices or he is not, based on his role within the

    organisation. For back office employees that need not be such a problem, but for front office

    employees and customers this is inappropriate. For them, the primary determinant of what they

    should be permitted to see and do is the slice across all systems and databases of the information

    concerning the customer at hand. A brief scan of information security literature is sufficient to

    establish that information security consultancies havent got this point quite yet: for every reference

    to slice-based access security there are thousands to role-based access security. On the positive side,

    there are emerging concepts which go a long way toward implementing the kind of access security

    that makes full use of the advantages that SOA has to offer: claims based access control, federated

    identities and de-perimeterisation.

    Make it manageable

    Rationale

    To get maximal benefits from SOA, it is insufficient to merely adopt new rules of behaviour. The way

    the organisation thinks about its self and its IT needs to change too. Otherwise the current

    organisation will revert to the old behaviour, not because it wants to but because it sees that as the

    only way to maintain control over its own agenda. To counter this, you must start with the way

    projects are organised.

    10. Dont leave infrastructure until last

    Why worry about infrastructure?

    SOA is all about separating infrastructure from business, not just in the functional sense, but also in

    the project management sense: a project manager should be able to concentrate on the business

    added value of the services that he is responsible for developing, without having to worry about the

    infrastructure. It is no less absurd that a business project manager should have to organise a new

    authorisation service as part of his project, than that he should have to install a new telephone

    exchange. To burden the project manager with such things results in an increase in the number of

    things that he has to coordinate, which results in turn in a disproportionate increase in the

    complexity of his task. Even letting him wait until some other project manager delivers the

    infrastructure is something that should be avoided where at all possible.

    The above remarks apply equally well to the hardware infrastructure as to the infrastructural

    services. The computing power and network capacity should always exceed the requirements, so

    that project managers who implement new services shouldnt have to worry whether the

    infrastructure can support a thousand service requests more per day than originally envisaged.

    Given that software development is typically an order of magnitude more expensive than IT

    infrastructure, the last thing we want is that systems development projects get bogged down by

    bureaucratic processes concerned with ensuring the availability of IT infrastructure.

  • Ten SOA commandments

    Author: Hans Wierenga Page 23 of 24 23-02-2010

    Infrastructure: how SOA is different

    The good news is that the SOA approach of saying what you want and then shutting up results in

    simple, stable interfaces to infrastructural services. That is why it is not necessary to implement the

    worlds most advanced infrastructural services before you can start any business projects. The

    complexities of these services lie not in the interfaces which are used to invoke them, but in the

    information which they request from other sources or maintain themselves in order to produce

    an optimal result. It is therefore perfectly feasible to start off with rudimentary infrastructural

    services, and to make these available for test and production use in plenty of time for them to be

    used by business projects.

    The bad news is that SOA results in much more functionality being classified as infrastructure. In

    particular, each chore results in a generic service that must be implemented as part of the

    infrastructure instead of being incorporated into each of the business functions for which it is

    required. That is not so bad by itself, but it does result in a new problem: how to organise things so

    that, for each part of the infrastructure, there is somebody with both the mandate and the resources

    to ensure that it is available on time with the appropriate functionality and capacity. An organisation

    that is not geared to addressing this problem is not geared to implementing SOA its as simple as

    that.

    Where do we go from here? Although the ingredients of SOA have been around for a while, SOA is radically new. Like the

    shipping container, it is more than just a new way to do the same as we have always done, because

    it enables us to cooperate on a new scale. And it is not primarily a new technology, but a new way of

    thinking. It is so different to the current way of thinking that you should not be embarrassed if you

    have to read through this article several times before it starts to sink in. It may take a while before

    your natural tendency to think that the suggestions it makes are impractical or unnecessary or both

    subsides sufficiently for you to be able to take stock of it all.

    To be fair, there will be situations in which the SOA solution is not directly practical for you. For

    example, you may acquire all your systems as packages from vendors who have no particular

    interest in applying SOA in the manner described in this white paper. In that case, treat the

    suggestions in this paper as a compass: use them to decide whether the choices you make are

    leading you in the right direction.

    Because SOA is so radically new, not all the concepts, tools and standards which we need in order to

    make full use of it are available just yet. Current gatekeepers do not do everything we should expect

    of a SOA gatekeeper, and current user interface facilities have difficulties with encapsulated XML.

    Worse, as this white paper has argued, there is currently much in the SOA world that is confusing,

    unnecessary or wrong. Current applications of SOA concepts and technologies do not get anywhere

    near achieving the level of interoperability that it is capable of. Too often we are confronted with

    interoperability problems that are due to somebody else having chosen to implement a slightly

    different subset of the many WS-* protocols that are on offer. The millions of possible solutions as

    to how parties communicate have been reduced to a few handfuls, but that is still far too many.

  • Ten SOA commandments

    Author: Hans Wierenga Page 24 of 24 23-02-2010

    But we have made a good start. The rest we can work out along the way. That, too, is SOA. We dont

    have to know what we cant know yet in order to make progress. An enormous potential to make

    things simpler, more reliable, more predictable, and functionally richer awaits us. It is time to start.