Case Study - Amazon S3Study+-+Integrati… · 2 Case Study: Integrating BizTalk and Lagan CRM...

Connected Systems Consulting Ltd

Case Study Integrating BizTalk & Lagan CRM

1

Case Study: Integrating BizTalk and Lagan CRM

Contents Introduction ....................................................................................................................................... 2

About Connected Systems Consulting ............................................................................................... 3

About the Authors .............................................................................................................................. 4

Michael Stephenson....................................................................................................................... 4

Contributors ................................................................................................................................... 4

Sedleigh Adams ......................................................................................................................... 4

The Project ........................................................................................................................................ 5

Batch Integration with Lagan .......................................................................................................... 5

Lagan Requests Data from other Applications ................................................................................ 5

Lagan sends updates to other applications ..................................................................................... 6

Project Use Cases ......................................................................................................................... 6

Project Architecture ........................................................................................................................ 6

Interesting Architecture Topics ........................................................................................................... 7

Managing where the customer record came from ........................................................................... 7

Load Levelling ................................................................................................................................ 8

Dynamic Error Handling ................................................................................................................. 9

2


Introduction Lagan CRM is a software product owned by Kana and is very popular in the UK in the public sector

and social housing sector. Over the years we have come across Lagan on a few occasions and have

decided to write a small paper about how we helped a customer who for the purposes of this paper

we will call Acme to integrate Lagan with their internal line of business applications.

In this paper we will focus on the BizTalk and integration perspective of this project. There were a

number of other areas of this project such as telephony integration and user experience which are

interesting topics in their own right but are not areas we will be covering in this paper.

3


About Connected Systems Consulting Connected Systems Consulting Ltd is a small UK based consultancy specialising in integration

solutions based on the Microsoft technology platform. Connected Systems Consulting has worked

with customers to develop on premise integration solutions based around BizTalk and cloud to on

premise integration solutions using the Microsoft Windows Azure Service Bus.

For more information about Connected Systems Consulting please refer to our website:

http://connectedsystemsconsulting.sharepoint.com/

http://connectedsystemsconsulting.sharepoint.com/

4


About the Authors

Michael Stephenson Michael Stephenson is the Managing Director of Connected Systems Consulting Ltd. Michael is also

Microsoft BizTalk MVP for a number of years and has been involved with the Microsoft Business

Platform Advisors programme including Windows Azure Advisors and Connected Systems Advisors.

Michael specialises in integration architecture and working with customers to help deliver effective

integration solutions based on the Microsoft platform.

Michael also specialises in ALM processes for integration projects and helping customers to have an

effective way of doing integration projects.

Michael is heavily involved with the UK Connected Systems User Group. To find out more about

Michael please refer to the following links:

Blog: http://www.geekswithblogs.net/michaelstephenson

Linked In: http://www.linkedin.com/in/michaelstephensonuk1

Contributors The following people have provided contribution to this case study.

Sedleigh Adams Sedleigh Adams is an experienced system integration tester and support operator having extensive

experience with testing of complex systems such as British Airways reservation systems and airport

security processes. More recently Sedleigh has been working with Connected Systems Consulting to

help develop a managed service offering for BizTalk customers. Sedleigh has worked with customers

to ensure that they are effective at progressing their BizTalk projects through various parts of the

project life cycle and helping to provide the transition of projects into business as usual which may

involve hand over to the customers own support teams or providing support directly from Connected

Systems Consulting.

http://www.geekswithblogs.net/michaelstephenson

http://www.linkedin.com/in/michaelstephensonuk1

5


The Project The project making up this case study involved Acme implementing Lagan CRM as a part of their

application architecture with the specific purpose of using the case management functionality. Acme

were interested in a sub-set of their customers who’s accounts had some issues with them and Acme

wanted to provide a new call centre with the capability of working with their customers to resolve

account issues which would in turn have a significant impact on Acme’s cash flow.

To allow the advisors in this new call centre to have the right capability to case manage these

customers they would require the following core integration requirements with Lagan.

1. Batch Integration with Lagan to populate the case queue

2. Lagan requests data from other applications to help advisors to review the customer details

3. Lagan sends updates to other applications upon processing of a case

Acme was also undergoing a technology upgrade project and had previously chosen to use Microsoft

BizTalk Server as a core part of their integration platform which would help this CRM project but also

support on going integration requirements for future projects.

Acme had engaged with Connected Systems Consulting as a partner to help with their BizTalk

implementation.

Batch Integration with Lagan Each week Acme would re-evaluate their case management position with a number of their back

office applications deciding which customers should be put into the case management process. The

back office applications would then produce a number of batches containing information about the

customers who needed case management and would submit them to BizTalk. The aim was that after

the batch process the back office applications and Lagan CRM would be synchronised and the start

of that week the call centre advisors would have prioritised lists of customers who they could contact

to help work through any account issues.

In the first phase of the batch integration there would be two batches of around 1000 records which

would be submitted for the pilot but overtime this would be rolled out across Acme’s other business

areas and by the end there would be up to 100 batches received within a very short time scale.

Lagan Requests Data from other Applications When the call centre advisors were to begin work on Monday their queues would be full of customers

who needed case managing. As an agent in the call centre would begin working on a case they

would use Lagan to process the customer’s record and to record interactions such as outbound calls

to the customer.

Lagan initially would not store all of the information for the customer and in the case at Acme, Lagan

was not used as the master data source for customer information. The core customer information

was held in the back office applications and Lagan played the role of acting as a case management

front end to a lot of this data. Because Lagan would need to request additional information from other

back office applications BizTalk was used to facilitate this integration.

An example of this part of the process would be if the call centre agent needed to see some of the

customers historical interactions with Acme then Lagan would request this data and BizTalk would be

used to source this data from the application the case was originally loaded from.

6


Lagan sends updates to other applications In the same way that Lagan could obtain additional data from the back office applications a call centre

agent may also update some details about the customer and when this was committed then Lagan

would send this update to BizTalk and BizTalk would ensure that the right source application was

updated appropriately.

Project Use Cases The following use case diagram illustrates the integration use case requirements for this project.

Project Architecture The below diagram illustrates the integration perspective on the project architecture.

uc Primary Use Cases

System Boundary

Batch Case

Synchronisation

Back Office

Application 1

Request Additional Data

Back Office Application 2

Lagan

Update Data

«flow»

«flow»

7


Basic Clustering & Load Balancing

Back Office Application A

Back Office Application B

Lagan Web Server

Lagan Database Server

Call Centre Agent

Desktop

Interesting Architecture Topics In the next section we will discuss some of the interesting architecture topics from this project.

Managing where the customer record came from One of the areas of architectural interest concerns the routing between applications when Lagan

wants to query some additional data for a case. Sometimes you may consider a master data registry

solution but Acme do not currently have a master data management solution so the solution which

had been implemented involved telling Lagan which source application a case came from when cases

are originally created from the batch process. This data is then flown with the requests for additional

8


data and BizTalk uses this data attribute to implement content based routing to go back to the original

source application to retrieve additional data.

This actually made the routing and publish subscribe pattern for the messaging between applications

a lot simpler as Lagan was able to act as the register of where the customers master data was held.

The below diagram illustrates how the source application is flown from Lagan and then used by

BizTalk to implement content based routing through property promotion.

MESSAGE BOX

PublishPublish

SubscribeSubscribe

Back Office Application A

Back Office Application B

Lagan Web Server

Message submitted

containing source

application from Lagan

which is promoted to

BizTalk context

Filter for back office

application B

Eg. Source Application

= “Back Office

Application B”

Load Levelling One of the key roles that BizTalk plays is during the batch processing features. In this area there will

be a queue of 1000’s of updates that need to be applied to cases in Lagan. In the Acme scenario

they are running an active/active instance of Lagan middle tier web services and the number of

updates which need to be applied could easily overload the web server and affect user activity if they

were sent to Lagan as fast as BizTalk would be capable of sending them.

This customer did not have the Lagan JMS capabilities available and all updates were to be applied

by the web service API. BizTalk has a number of ways of throttling the load and we were able to

control the load so that the through-put was throttled to a point that balances the duration to load the

batch with the speed the users need to work through the queues of resulting cases.

As the number of batches increases this throttling pattern simply means that the queue in BizTalk

gets a bit longer and we are feeding cases into Lagan at a constant rate that the Acme setup can

comfortably handle.

The below diagram illustrates the throttling implementation.

9


MESSAGE BOX

PublishPublish

SubscribeSubscribe

Debatching Orchestration

The debatching

orchestrations will

handle many

concurrent batches

and queue up 1000's

of messages into the

BizTalk message box

waiting for processing

Adapter

Send to Lagan Lagan Web Server

Throtting patterns can

be used to control how

many messages are

sent at once and the

rest are queued in the

BizTalk database

Debatching Orchestration

Dynamic Error Handling One of the interesting areas of the batch process was handling of different types of error. BizTalk

comes with a number of out of the box error handling features such as automatic retries and retry

intervals but one of the most useful in this scenario was the failed message routing capabilities.

When we are sending errors to Lagan in the batch process we implemented an orchestration which

would subscribe to failed messages from the batch send ports and this would allow us to create a

more complex handling routine. Out of the box the automatic retries do not look at what type of error

has occurred and simply implement one type of error handling regardless of the error type.

In our error handling orchestration we wanted to handle errors as follows:

Error Action

Lagan web server is down and returns a web exception

We would wait for a short period of time and then send the message for retry. The retry count would be incremented. If this error was persisting and retries were exhausted we would suspend the message and wait for an operator to resume

10


it.

Lagan throws an error with Error Code 21 In this case we need to seek some assistance from the Lagan support team so the message is suspended and the BizTalk operator can query with the Lagan team how to proceed. If the message can be processed then the BizTalk operator can resume the message and it will load into Lagan.

Lagan throws an error with Error Code 12 As above

Lagan throws a SQL Deadlock error In this case the error can usually be sent for retry usually works the next time. In these cases we often use diagnostics tools to inform the Lagan team when instances such as these have occurred so they can be investigated.

Lagan returns a case is already closed error This case identified a defect in the process where there was a mismatch between some status codes. The case from the back office application was identified as already being in Lagan and that it could be closed but the status was already closed. Initially our process would get these and execute the default error handler but once we identified this issue we were able to change the error handling rules at runtime to clean up these messages until the issue was fixed.

Default Any other error which had not already been matched would be suspended and the BizTalk operator would be able to review the error and decide which action to take and then send the message for retry if required.

To implement this dynamic error handling capability in BizTalk we used the BizTalk Business Rules

Engine to develop a policy which would inspect the error details and the message which we were

attempting to process. The rules policy would then workout which action should be taken. The

actions that were supported include:

1. Increment the retry count and send the message for retry

2. Wait a period of time then increment the retry count and send the message for retry

3. Delete the message as it will never be able to be processed but raise an alert to inform the

operator

4. Delete the message as it cannot be processed and is a known issue that can be cleaned up

5. Suspend the message and raise an alert to the BizTalk operator who will decide if the error

can be sent for retry or not.

Conclusion In this paper we have tried to provide a high level overview of a successful project involving BizTalk

and Lagan to deliver significant business value to Acme. Connected Systems Consulting were

approached by Acme to help when the project found some difficulties during the early stages and

11


Connected Systems Consulting were able to help Acme to put the right team and ALM processes in

place alongside a simplified architecture which ensured the project was successful.

As a follow up to this case study we will also produce some guidance to help people with some of the

lessons learnt through integration with Lagan.

Case Study - Amazon S3Study+-+Integrati… · 2 Case Study: Integrating BizTalk and Lagan CRM...

Documents

Transcript of Case Study - Amazon S3Study+-+Integrati… · 2 Case Study: Integrating BizTalk and Lagan CRM...