A Software Development Methodology for EAIhosteddocs.ittoolbox.com/ER011405.pdf · A Software...

A Software Development Methodology for EAI

WORLD HEADQUARTERS5295 Hollister

Houston, TX 77040 USAtel: 832.590.6100

toll free: 1.888.225.3139fax: 832.590.6161

www.zettaworks.com

Copyright © 2004 ZettaWorks LLC. All rights reserved. This material is or contains Proprietary Information,

Confidential Information and/or Trade Secrets of ZettaWorks LLC. Disclosure to third parties and or any

person not authorized by ZettaWorks LLC is prohibited. Use may be subject to applicable non-disclosure

agreements. Any distribution or use of this material in whole or in part without the prior written approval of

ZettaWorks LLC is prohibited and will be subject to legal action.


The Opportunity

EAI has a proven track record of achieving quick ROI. Success stories of amazing ROI for EAI projects

abound. Gartner found, “As much as 30 percent of the costs associated with implementing major

packaged applications will be consumed by integration. Use of an [integration] broker will reduce these

integration costs by one third. During maintenance, when a single change to an application can have a

rippling effect on several dozen interfaces, use of a broker can reduce costs by two-thirds.”

While the application integration cost savings can be significant, the new architecture of application

integration is driven by organizations seeking competitive advantage. The value transformations

achieved though the automation and integration of an organization’s business processes creates razor

sharp competitive advantage. In addition, companies are extending their business processes through

the value chain to customers and suppliers via the Internet.

The Challenge

While many companies are reporting triple-digit ROI on EAI projects, some have experienced less

than stellar returns or outright failures. This dichotomy of benefits realized is not uncommon for new

technology adoption. EAI is evolving with the continuing introduction of standards (such as Web

Services) and new entrants (application server and pure-play Web Services vendors). The technology

risk of EAI is particularly challenging due to the following factors:

• Organization change in necessary since EAI crosses system boundaries

• The architecture is enterprise in scope encompassing dispersed and heterogeneous systems

• The infrastructure is distributed requiring high availability and scalability

• The project life cycle methodology requires changes due to complex system dependencies, EAI

specific design patterns, and the change impact to the infrastructure and users

• Program management is often complex due the project scope, interdependencies and new

technology risks

• Quality assurance is difficult since EAI components are distributed, have many interfaces, require

new testing environments, and message based testing tools

• New competencies must be developed spanning project management, analysis and design,

development and operations.

A Framework for EAI Success

This paper presents a framework for overcoming the challenges associated with enterprise scale

integration projects. Given that it is difficult for an EAI effort to drive the change of an organization’s

Software Development Lifecycle (SDLC), a pragmatic approach is required with flexibility to change

approach and deliverables. To this end, an analogy is made with modern iterative methodologies where

applicable to EAI projects.

Successful EAI projects are not all about technology, defining the business goals and establishing a

working organization structure are the key first steps. We will examine an EAI methodology from the

perspective of achieving business, organizational and technology goals as follows:

• Establish an Integration Competency Center (ICC) staffed with a cross-function technology team

• Develop a formal EAI business case with strong executive sponsorship, objectives in business

terms and a target Return on Investment (ROI)

• Select EAI tools matching business and architecture requirements to product features and

functions

• Develop an Enterprise Architecture Blueprint based on EAI design patterns and best practices

that defines the EAI logical and physical architecture

• Adapting the current architectures and methodology in use to adjust for EAI – a OOA/OOD

approach with specific EAI design patterns

• Managing iterations of the EAI project lifecycle from analysis to operations

Ideally, an EAI Methodology should improve upon traditional methodologies by addressing specific

issues related to EAI, as well as including best practices, standards, design patterns and processes.

Establishing the Integration Competency Center

Establishing the Integration Competency Center (ICC) is the first step in developing an EAI solution.

Gartner says, “Creating a single organization body that brings the diverse skill set together helps create

the focus, momentum, critical mass, processes and standards required to build and execute world-class

application integration strategy”. Ideally, the ICC should be formed early to define the strategic EAI

business case that drives the software selection process. This would improve the software selection

results and produce an EAI roadmap early avoiding a lag time between software purchase and

implementation improving ROI.


The ICC seeks to solve the cross disciplinary problems associated with EAI – pulling a team together

with differing critical skill sets. The ICC also provides the role of EAI governance, which applies to project

selection and portfolio management as well as technology standards adoption and management. These

standards include architecture, coding standards, design patterns, reusable components and the Common

Information Model (CIM) – canonical data representations.

Key steps to create and manage the ICC include:

• Create a core ICC team representing business and technical stakeholders prior to software

acquisition to drive the strategic business case, project roadmap and software acquisition

• Establish a Program Manager within the ICC, along with governance policies to manage complex

project dependencies

• Define the roles and needed competences, and create the staffing and training plans to establish

competency

• Address IT cross functional team project requirements early, through the ICC team members, such

as establishing EAI quality assurance policies and procedures, and plans for operations support of

infrastructure monitoring and management

• Establish the baseline EAI architecture

• Within the EAI methodology, create a closed loop process for updating architecture, design patterns,

the CIM, etc., with the governance model within the ICC

The EAI methodology becomes the means for the ICC to define and refine processes such as physical and

logical architecture, project initiation and justification, operational service level agreements, etc.

Developing the High-level Business Case

Many companies take a rudimentary approach to EAI project justification by simply calculating the labor

savings of EAI versus point-to-point integration. However, looking only at cost savings misses opportunities

for the dramatic business process improvement EAI can deliver. EAI is an enabling technology with the

following potential benefits:

• Optimization of assets with real-time logistics management

• Increased inventory turns through B2B integration

• Increased sales by reducing stock outs with real-time inventory management

• Accelerated the integration of mergers and acquisitions


• Improved visibility of financial data by integrating multiple ERP systems

• Achieved process improvements with Business Process Management

• Improved productivity and access to information with the use of integrated portals

• Enhanced customer service by integrating systems to web based applications

To uncover less than obvious EAI benefits, it is necessary to examine business objectives, applications and

system architecture to develop a high-level business case and feasibility study for EAI. The quickest way to

execute this strategy is through a set of facilitated sessions focusing on business objectives and IT’s ability

to deliver systems that keep up with key business drivers. These sessions examine the strategic value of IT

assets looking for stovepipe applications that act as barriers to the ability to achieve the corporate mission.

The goal of these sessions is to develop a portfolio-level understanding of how the existing and planned

systems meet the organizational goals, and how EAI could accelerate the realization of these goals. The

business case deliverables are a feasibility study covering the following topics:

• Definition of key EAI benefits

• Target applications for integration – a potential project roadmap

• IT organization impact

• High level ROI potential

• ‘Go’/’No Go’ recommendation for a formal EAI product evaluation

EAI Product Selection

Once EAI is determined to be feasible, a formal EAI product selection process should be performed.

Many EAI project are doomed at onset by selecting the wrong EAI suite for the enterprise. EAI suites

have proliferated, ranging from low-end data transformation tools to enterprise class product suites,

including components such as messaging-oriented middleware, integration broker, portal, business activity

monitoring, and monitoring and management. There is a dizzying array of features and functions, emerging

technologies and literally hundreds of vendors professing EAI solutions. And, just picking the market leader

can be problematic since the emerging technologies are often offerings from new players.

EAI suites must fit within the target architecture and support the end applications to be integrated in the

most efficient way. A vendor can have a great EAI suite, but if a single adapter lacks necessary functionality

for a critical business application, the project is at great risk of failure. Also, the physical architecture

features, such as platform support, scalability and fault tolerance can also become major implementation

issues. Likewise, vendors have strengths and weaknesses that must be measured against requirements.


The following outlines a five step EAI selection process (figure 1) to address the challenging task of EAI

software selection:

1. Architecture and Needs Analysis – Joint Application Design (JAD) sessions to determine

differentiating criteria

2. EAI Business Case Development

3. ROI-Based ‘Go’/ ‘No Go’ Decision

4. Request For Proposal Process

a. Complete and Issue RFP

b. Verify Vendor References

c. Refine Vendor Short-List

d. Facilitate Scripted Demos & Proof of Concept Reviews

e. Create Decision Matrix

f. Cost Negotiations & Final Selection

5. Recalculation of Total Costs (Including Implementation) & ROI

The Architecture Blueprint

It is beneficial at this point to consider the impact of existing methodologies in use with respect to the

creation of EAI specific architecture. The Rational Unified Process (RUP) has a more structured approach

(many would argue heavy) to architecture than agile methods such as eXtreme Programming (XP). XP

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


proposes quick iterations and frequent refactoring (incremental improvements to internal design) as an

alternative to formal architecture. The concept of refactoring applied to EAI when considering the potential

ripple affect across multiple interfaces when changing things like the CIM or standards such as coding,

message naming or queue topics becomes unwieldy.

With EAI it is necessary to understand the big picture of the target enterprise-level integration for

architecture. This leads to the recommendation to create baseline architecture in the RUP style – creating

candidate architecture in the inception phase and completing the baseline in the elaboration phase. This

works particularly well when a reference architecture for the EAI product already exists (as a vendor or

integrator’s best practice) and needs site specific changes and vetting with a pilot project.

This is not to say that architecture is not an iterative process. Architecture should evolve by continuously

adding to the CIM, design patterns, interface design artifacts, and adjusting infrastructure – within the

governance of the ICC. However, it is necessary to have a starting point for standards and a strategy to

fulfill non-functional requirements on the enterprise scale. The RUP architecture centric approach proposes

an iterative architecture, addressing the most architecturally significant use cases first, which works well for

EAI.

Having addressed the concern of EAI architecture process, it is necessary to consider information

organization and actual deliverable content. With respect to organization, Philippe Kruchten, in his

paper appearing in IEEE Software titled “Architecture Blueprints – The “4+1” View Model of Software

Architecture”, presents an architecture model consisting of concurrent views specific to various

stakeholders as follows:

• Logical view for end-user functionality

• Development view for programmers and software management

• Process view for integrators dealing with performance and scalability

• Physical view for systems engineers dealing with topology and communications

The precept of architecture views is particularly interesting regarding the introduction of EAI to an

organization as a new enabling technology. The organization of the architectural deliverables can target

specific stakeholders to decrease the learning curve. To this end, a set of architecture guides (specific to

enterprise architectures, developers and systems administration) address the typical EAI stakeholders (the

end user or logical view would be expressed within a project specific architecture – not at the enterprise

level). Using this scheme, Kruchten’s architecture model views of development, process and physical are

addressed within the architecture guides relative to their specific stakeholders.


Given the organizational model of architecture, developer and systems administration the architecture

deliverables follow:

Architecture Guide (deliverable relationships in figure 2)

• Physical and logical design

• High-level as-is and to-be interface diagram

• EAI product roles within the organization

• Common application services – e.g. auditing, error handling, security

• Non-functional requirements and strategy to meet requirements – e.g. fault tolerance, capacity

planning, performance

• Infrastructure – network and platform configurations

Developers Guide

• Naming standards

• Product usage guidelines

• Common design patterns

• Guidelines for reuse

• Change management strategy

Systems Administration Guide

• Build documents

• Monitoring and management overview – e.g. logs, consoles, scripts, admin tools

• Environment management – change control and migration

• Run time component inventory

• Directory structure and security

The ideal scenario for creating the EAI architecture is through a series of JAD sessions with the target

stakeholders and an EAI vendor or integrator’s Subject Matter Expert (SME). The SME should have a set of

best practices and candidate architectures that can be customized to fit the specific environment based on

the JAD session results and analysis.


The Software Development Lifecycle Implications

The strengths of a modern SDLC comes into play during the analysis, design and construction phases

of an EAI effort. To consider the use of an SDLC for EAI, let us constrain the definition of EAI to include

building A2A and B2B integrations and not expand the scope to include Business Process Management

(BPM), portals, Business Activity Monitoring (BAM), and whatever else EAI vendors decide to throw into

their software stack.

The EAI effort then can be viewed as a subset of a larger effort to accomplish a business goal such as

building a composite application or a managing a business transaction using BPM with system integrations.

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


The larger effort may include user interfaces, portals, BAM, etc., but the EAI effort would be focused on

the system-to-system interfaces. The intent of defining EAI in this context is to focus on the process and

deliverables needed in this specialized area – purposely avoiding well understood topics such as user

interface design and business process modeling.

A2A integrations are a subset of the overall business process where the EAI architect analyzes the

interfaces and messages with fine granularly, but must understand the overall business process from the

perspective of transaction integrity, security and performance. This requires an understanding of the big

picture, but this can easily be accomplished by reviewing system level deliverables such as use cases and

activity diagrams.

By focusing on interfaces and messages EAI becomes analogous to OOA/OOD in that entire applications

are abstracted as objects. The goal is to create composite applications consisting of multiple applications

with messages flows between the applications interfaces (or adapters). The logical design objectives for

EAI consist of the following activities:

• Define the business domain to be integrated (e.g. new customer add)

• Define the integration points within the domain (e.g. a SAP adapter)

• Define the message flows and sequences between the integration points

• Define the message formats necessary for A2A interaction

• Identify the components that will be used for common services such as data mapping and error

handling, the message formats and component interactions

• Define the control logic necessary for message routing, transformations, etc.

• Define the test cases for the integration

The EAI design goal is to produce lightweight deliverables describing the interfaces, events and message

flows with an iterative methodology. UML proves to be well suited for EAI design artifacts since EAI treats

applications as objects and UML artifacts express object interactions. Consider the UML models and

deliverables in figure 3 in the context of the logical design deliverables for EAI as follows:

• Context diagram – defines the integration domain boundary

• Use cases – functional requirements

• Collaboration diagrams – component responsibilities and how they collaborate

• Sequence diagrams – message flows within the integration

• Activity diagrams – process flows through distributed EAI components


The context diagram sets the logical boundary of the integration, essentially decomposing the goal of

EAI into iterative steps such as a business process, data element(s) or geography. The concept of an

integration domain is critical: Without decomposing the complex problem of EAI into manageable domains,

the problem becomes untenable. For example, consider the problem of defining the canonical form of a

customer in a global organization with hundreds of businesses and systems all over the world. One must

decompose this problem into domains with a domain integration strategy.

Within the context of the domain, use cases define functional requirements. Test cases are part of the

requirements that drive system design; users, analysts and developers must understand them. Test cases

are derived from use cases and event flows.

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


Collaboration diagrams are important to EAI to define the various components and their roles in the

integration – e.g. adapters, processes within brokers, mapping etc. Sequence diagrams further define

the event driven nature of EAI by depicting the message flows between EAI components in time-ordered

sequence (arguably the most valuable deliverable). Activity diagrams are also particularly useful for

showing the division of responsibility of an integration process across distributed systems in swim lanes.

The domain of the integration, as defined by the context diagram and use cases, establishes the domain

model for data integration. This is essentially the mapping of multiple data sources either point-to-point

between applications or to a canonical form. A rule of thumb is to use a canonical representation if five

or more systems will require the data. Canonical data formats then become part of the CIM. A simple

definition of the CIM (for brevity) is a common externally-managed repository of data semantics that

promotes reuse. The CIM is analogous to the vocabulary for future integrations and the UML interface

specifications (such as the sequence diagrams) are the way to communicate that vocabulary. The CIM and

interface specifications are critical for reuse and should be stored in an external repository – not buried in

broker process and mapping logic.

The notion of decomposing EAI to interface level granularity without introducing a higher level methodology

for a more holistic view of the end-goal of a totally integrated system seems counterintuitive. However,

while it is possible to understand EAI holistically at the interface level, it is difficult, if not impossible, to get

agreement on the desired holistic state of a totally integrated enterprise at the business processes level.

And, if you were able to capture all these business processes, it would simply be a snapshot in time since

business processes evolve quickly – EAI enables process agility. It is more affective to view EAI at the

application portfolio level with a strategic approach to improving high-value processes in priority sequence

– a role of the ICC. The methodology goal is to develop an adaptive EAI architecture using the following

techniques:

• Decomposition into integration domains with a domain integration strategy

• Encapsulate applications as objects with well defined interfaces to achieve low coupling

• Expand on the CIM with each integration domain, building the semantics of the enterprise iteratively

• Build the repository of interface definitions and message interactions for reuse with each integration

iteration

• Build the repository of design patterns (technical and business) for reuse

• Assemble composite applications based on application objects using orchestrated processes visually

modeled in the EAI tools

We have examined EAI Methodology in terms of a process and modeling language (UML). UML is the

graphical models that a methodology uses to convey system design. The methodology is a process that

defines the steps necessary to complete the analysis, design, construction etc.


There is a continued debate on the merits of the RUP versus agile methods such as XP. While it is

certainly possible to run EAI projects in an XP fashion there is concern that the lack of rigor can lead to an

incomplete or inconsistent process around integration domains, the CIM and interface design specifications.

Since these deliverables become the basis for future integrations, process consistency is critical. We will

therefore look at a lightweight (i.e., consisting of few UML deliverables) instance of RUP as the candidate

EAI methodology – RUP’s iterative nature and focus on architecture early in the process are particularly

well suited to EAI.

The RUP architecture is defined in two dimensions as shown is figure 4. You progress through a project

timeline through iterations of the RUP defined phases. Within each phase the disciplines of RUP are

performed in varying degrees of effort as shown by the horizontal graphs. To consider the use of the RUP

for EAI we will map the RUP phases to EAI specific tasks and disciplines.

The RUP inception phase has the objectives of establishing the project vision, system functionality,

candidate solution, costs and schedule, and stakeholder final approval on the vision cost, etc. With respect

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


to EAI the RUP inception phase consists of the following activities:

• The ICC is engaged to prepare the EAI candidate solution

• A succinct project vision describes the problem, vision of the solution, stakeholders and high-level

use cases

• Non-functional requirements are assessed for architecture impact

• Artifacts are examined for reuse: CIM, interfaces specifications, components, etc.

• Candidate solution is proposed in terms of architecture, interfaces and EAI component roles

• The preliminary schedule and cost estimate is produced for approval

The RUP elaboration phase RUP has the objectives of creating detailed project requirements, the plans to

mitigate risks, the baseline architecture and refine project costs and schedule. A RUP elaboration phase for

EAI consists of the following activities:

• Create a risk list and action items to mitigate – e.g. technology pilot

• Complete UML deliverables for domain and interface design (figure 3)

• Complete domain specific message specifications reusing the CIM (figure 3)

• Detailed design of CIM, interfaces and configurations

• Define test strategy, plans and cases (figure 3)

• Create baseline architecture to meet non-functional requirements (figure 2)

• Refine project plans and costs

The RUP construction phase RUP has the objective of code construction while minimizing costs, creating

parallel tasks, and iteratively developing the complete solution. A RUP construction phase for EAI includes

the following activities:

• Create environments for test, QA and production

• Component builds to specifications and architecture

• Unit, integration and system test

• Stress and operational test – fault tolerance, capacity

• Create migration and build documentation

• Training for support and system administration

• Create monitoring and management tools

The RUP transition phase consists of user beta testing, user and support staff training, deployment and

user signoff. Also, the RUP documents lessons learned for future iterations. Specific to EAI, the RUP

transition phase consists of the following activities:



• Deployment and acceptance test

• Production turnover and user signoff

• Change management

• Environment management and operational support

• Update the CIM and interface repository

• Update design patterns and common components

The complexity of the RUP can be intimidating. However, one can construct a lightweight RUP instance for

EAI describing task flows, inputs, roles and deliverables in a single document – without buying the RUP

software. Also, the suggested UML deliverables can be produced with inexpensive modeling tools like Visio.

If the RUP is a corporate standard, configuring an EAI instance is straightforward.

Conclusion

The intent here is not to turn an EAI methodology into a study of academic purity or a religious crusade

(where zealots certainly abound) for an ideal approach, but to produce a practical and proven guide to

adapting EAI to existing practices. There is no perfect mapping of EAI to either the RUP or agile methods

like XP – especially in the early strategic stages suggested herein. Coming from the perspective of an

outside consultant, one must adapt to the environment to be successful with EAI. The approach outlined is

adaptable, lightweight, quick and, best of all, it simply works.

Takeaways

Business TechnicalA proven EAI methodology reduces risks and improves ROI

The Integration Competency Center should create a cross-functional team to provide the expertise, gover-nance and standards necessary for successful EAI.

The business case for EAI should explore opportunities for business process improvement and ways to generate revenue – not just the cost savings of improved integra-tion productivity.

The EAI architecture should be presented in multiple views – relevant to specific stakeholders.

An EAI methodology will create business process flex-ibility by allowing quick changes and the assembly of business processes from components.

The analysis and design deliverables for EAI should be lightweight using a visual modeling language such as UML to describe interfaces, events and messages.

The EAI methodology should be iterative with lightweight deliverables, but with enough rigor to develop and en-force enterprise standards and create an EAI repository for reusable artifacts.

Eric Roch is Vice President of Technology for ZettaWorks (a pure-play EAI consultancy), where he is responsible for

the development of EAI methodology and solutions. During his more than 20 years of experience in IT, he has worked

in roles such as executive-level management, technical architecture, and software development in leading technology

companies. Eric’s experience includes strategic planning and commercialization of methodologies and software, as well

as technical architecture for EAI and distributed applications.

e-Mail: [email protected]

About the Author


A Software Development Methodology for EAIhosteddocs.ittoolbox.com/ER011405.pdf · A Software...

Documents

Transcript of A Software Development Methodology for EAIhosteddocs.ittoolbox.com/ER011405.pdf · A Software...