UNIVERSITY OF WATERLOO 

Software Engineering 

 

 

AAnn  EEvvaalluuaattiioonn  ooff  WWoorrkkffllooww  FFrraammeewwoorrkkss  ffoorr  tthhee  

AAuutthhoorriizzaattiioonn  ffoorr  EExxppeennddiittuurree  SSyysstteemm  

 

 

 

 

Xyz Inc. 

Sometown, Somewhere 

 

Prepared by 

Some Student 

Student ID: 20000000 

User ID: sstudent 

3A Software Engineering 

September 5, 2006 

Some Student My Street My Town, ON A1A 1A1 September 5, 2006 Dr. William Wilson, Director Software Engineering University of Waterloo Waterloo, ON N2L 3G1 Dear Dr. Wilson:

The enclosed report, An Evaluation of Workflow Frameworks for the Authorization for Expenditure System, is my third work term report. It is related to my work at Xyz Inc., where I was employed during my 3A work term.

Xyz is a consulting firm that specializes in custom software development. I was

employed as a consultant, and worked on several client-facing projects over the course of the work term. One such project involved the design, development, and deployment of a human workflow solution for Client Company, a leading stuff producer.

One of the fundamental design decisions that I faced during the planning phase of

the project was choosing a workflow framework. This report evaluates two alternatives against multiple criteria and identifies the more appropriate choice for the project.

I would like to thank Dr. Andrew Morton and Santheeban Kandasamy for

suggesting the use of Multi-Criteria Decision Analysis (MCDA). I would also like to thank my colleague, Colleague, for his insight into the application of Microsoft BizTalk Server to human workflow systems.

I hereby confirm that I have received no help, other than what is mentioned

above, in writing this report. I also confirm this report has not been previously submitted for academic credit at this or any other academic institution. Sincerely, Some Student Student ID: 20000000 Encl.

iv

Executive Summary 

Client Company, a leading stuff producer, has engaged Xyz Inc., a consulting

firm, to design a human workflow solution to automate their expenditure approval

process. A key deliverable for the planning phase of the project involves a

recommendation of a workflow framework. This report provides a comparison of two

alternatives, Microsoft BizTalk Server 2006 and Microsoft Windows Workflow

Foundation (WF), assessing their suitability to the system.

This report evaluates the alternatives against a set of seven criteria: persistence of

long-running workflows, transparency into the workflow, flexibility for human-driven

processes, maintainability of business rules, extensibility, risk, and cost. Because of the

subjective nature of the criteria, this report leverages Multi-Criteria Decision Analysis

(MCDA) to provide a robust, quantitative analysis of the alternatives.

The two alternatives are similar in some aspects and very different in others.

BizTalk and WF have similar persistence and business rules mechanisms. BizTalk, as an

enterprise-class product, provides a more robust set of tracking tools, while WF is

considerably more flexible for human workflow. WF, which is freely distributable, is

much less expensive than BizTalk, but this factor is offset by the fact that, as pre-release

technology, WF is more risky as a technology investment.

Ultimately, the analysis indicates that WF is the more appropriate solution,

whether or not the cost factor is included. This report recommends that the architect

design a solution based on WF, that the development team undergo whatever training is

necessary, and that the project manager allocate additional time for deployment.

v

Table of Contents 

Executive Summary ........................................................................................................... iv List of Figures .................................................................................................................... vi List of Tables .................................................................................................................... vii 1 Introduction............................................................................................................. 1 2 Background............................................................................................................. 3

2.1 The AFE Workflow .................................................................................... 3 2.2 Design Constraints ...................................................................................... 4 2.3 Evaluation Criteria ...................................................................................... 5

2.3.1 Persistence....................................................................................... 6 2.3.2 Transparency................................................................................... 7 2.3.3 Flexibility........................................................................................ 7 2.3.4 Maintainability................................................................................ 8 2.3.5 Extensibility .................................................................................... 8 2.3.6 Risk ................................................................................................. 8 2.3.7 Cost ................................................................................................. 9

2.4 The Analytic Hierarchy Process ................................................................. 9 2.4.1 Overview......................................................................................... 9 2.4.2 Criteria Weighting ........................................................................ 10

2.5 The Alternatives........................................................................................ 11 2.5.1 Microsoft BizTalk Server 2006 .................................................... 11 2.5.2 Microsoft Windows Workflow Foundation.................................. 11 2.5.3 Alternatives Not Considered......................................................... 11

3 Evaluation ............................................................................................................. 13 3.1 Persistence................................................................................................. 13 3.2 Transparency............................................................................................. 13 3.3 Flexibility.................................................................................................. 14 3.4 Maintainability.......................................................................................... 14 3.5 Extensibility .............................................................................................. 15 3.6 Risk ........................................................................................................... 15 3.7 Cost ........................................................................................................... 16

4 Results................................................................................................................... 17 5 Conclusions........................................................................................................... 19 6 Recommendations................................................................................................. 20 References......................................................................................................................... 21 Acknowledgements........................................................................................................... 22 Appendix A: The Analytic Hierarchy Process.................................................................. 23

vi

List of Figures 

Figure 2.1: The AFE Workflow.......................................................................................... 4 Figure 2.2: Sequential Workflow........................................................................................ 5 Figure 2.3: State Machine Workflow.................................................................................. 6 Figure 4.1: Graphical View of Results ............................................................................. 18

vii

List of Tables 

Table 1.1: Authority Amounts ............................................................................................ 1 Table 2.1: Rating Scale ..................................................................................................... 10 Table 2.2: Criteria Weighting ........................................................................................... 10 Table 4.1: Weighted Scores .............................................................................................. 17 Table A.1: Rating Scale .................................................................................................... 23 Table A.2: Criteria Matrix ................................................................................................ 24 Table A.3: Normalized Criteria Matrix ............................................................................ 24 Table A.4: Comparison Matrix ......................................................................................... 25 Table A.5: Normalized Comparison Matrix ..................................................................... 25 Table A.6: Final Scores..................................................................................................... 25

1

1 Introduction 

Client Company is an international stuff doing company headquartered in

Toronto, Ontario. With the recent acquisition of A Company Inc., Client Company is

currently the world’s largest stuff producer [1]. The company operates 27 things across

five continents and trades on the Toronto, New York, London, Euronext, and SWX Swiss

stock exchanges.

Client Company’s spending policy mandates that an Authorization for

Expenditure (AFE) form is submitted for all cash outlays greater than or equal to

$10,000. Once submitted, an AFE is routed to a series of approvers. The AFE is

escalated up the organizational hierarchy until it is approved by an individual with

sufficient authority. An employee’s authority amount is determined by her position

within the company, as shown in Table 1.1*.

Table 1.1: Authority Amounts

Position Authority Amount

Chief Executive Officer (CEO) >$100X

Chief Operating Officer (COO) $100X

Designated Senior Executive $50X

Vice President $20X

General Manager – Category A Things $10X

General Manager – Category B Things $5X

General Manager – Category C Things $2.5X

General Manager – Category D Things $1X

Stuff does not have any system in place to automate the AFE workflow; AFEs are

routed by e-mail, by fax, or in person, while approved AFEs are archived in filing

cabinets. The client has identified several issues with this manual process:

* The absolute authority amounts are confidential, and have been replaced with relative authority amounts.

2

• Once an AFE is submitted, there is no transparency into the state of the

workflow. Often, AFEs accumulate on an approver’s desk while other

employees attempt to locate them.

• There is no electronic repository of approved AFEs. The filing cabinets

contain thousands of AFEs, and finding a specific one is extremely time-

consuming.

• Often, due to human error, an AFE is not routed to the correct approvers,

and does not obtain approval from an individual with sufficient authority.

• As a result of the various inefficiencies in the manual process, it takes far

too long to obtain approval for an AFE.

Client Company has engaged Xyz Inc., a consulting firm, to design, develop, and

deploy a software solution to automate the AFE workflow. One of the key deliverables

for the planning phase of the project is an evaluation of workflow frameworks and a

recommendation for the AFE project.

This report provides an evaluation of two workflow frameworks, assessing their

suitability to the AFE system. It first provides some additional background on the AFE

workflow. Next, it defines a set of evaluation criteria and provides a brief overview of

the alternatives. The report then evaluates the alternatives against each criterion. Due to

the subjective nature of the criteria, the report leverages Multi-Criteria Decision Analysis

(MCDA), a methodology for providing a robust evaluation of alternatives against

subjective criteria. Finally, the report summarizes the results of the analysis, draws

conclusions, and makes a recommendation.

The reader should have some background in enterprise software development.

Familiarity with MCDA is recommended but not required, as an overview of the

particular approach taken is provided in Appendix A.

3

2 Background 

2.1 The AFE Workflow 

The first step in the workflow involves the initiator preparing and submitting an

AFE form. Once an AFE is submitted, the system must analyze its contents and

determine the sequence of approvers; the business rules that determine this approval

chain are outside the scope of this report. The AFE is then routed to the first approver in

the approval chain. This approver receives an alert that her approval is required, reviews

the AFE, and submits her decision and comments. Each approver has the option to

approve or decline the AFE. If approved, the AFE is routed to the next approver in the

chain. If declined, the AFE is routed back to the initiator, who must revise and resubmit

the AFE. Once an AFE has been approved by each individual in the approval chain, it is

marked as approved, and the workflow is complete. Figure 2.1 provides a simplified,

conceptual model of the AFE workflow.

4

Initiator completesAFE form

Initiator submits AFE

Approve?

Final Approver?

Yes

No

Complete

Yes

No[route for revision/resubmission]

Initiator Route to next approver

Approver reviews AFE

Start

Figure 2.1: The AFE Workflow

In addition to the requirements described above, the system must be flexible to

accommodate exceptions to the normal process. As well as being able to approve or

decline an AFE, an approver should have the option to delegate to an alternate approver.

Furthermore, the system administrator should have the option to alter the approval chain

at runtime; for example, she may wish to bypass certain intermediate approvers and route

the AFE directly to the CEO.

2.2 Design Constraints 

Client Company has specified the following technical constraints for the AFE

system:

• It will be deployed to a single server.

• It must run on Microsoft Windows Server 2003.

• It must use either Microsoft SQL Server 2005 or Oracle Database 10g.

5

The workflow frameworks evaluated by this report adhere to these technical constraints.

2.3 Evaluation Criteria 

Workflow can be classified into two types: system workflow and human

workflow. The AFE process is a typical example of the latter. The two types are

fundamentally different and present different challenges.

In general, the participants in system workflow are software applications. The

integration of disparate software applications is often referred to as Enterprise

Application Integration (EAI). If this integration connects multiple businesses, the

workflow is often referred to as Business-to-Business (B2B) workflow. System

workflow typically executes in a rigid, predefined sequence. The process, rather than the

participants, dictates the sequence of actions. System workflow is often modelled as a

sequential flowchart, such as the one illustrated by Figure 2.2.

Figure 2.2: Sequential Workflow

6

As the name implies, the participants in human workflow are people. Although a

human-driven business process may leverage software applications, it is the human

participants that drive the workflow, not the software applications themselves. Human

workflow typically executes in a flexible, dynamic fashion, where participants often have

a choice of actions. The participants, rather than the process itself, dictate the sequence

of actions. This type of workflow is often modelled as a state machine, such as the one

illustrated by Figure 2.3.

Figure 2.3: State Machine Workflow

The following subsections define the criteria against which the alternatives are

evaluated. The criteria are specific to the AFE system, but most human workflow

systems have very similar high-level requirements.

2.3.1 Persistence 

Unlike system workflow, human workflow must be long-running and persistent.

Whereas a system process can take seconds, minutes, or hours, a process that involves

human tasks can take days, weeks, or even months. Furthermore, system processes often

execute from start to finish without interruption, whereas human-driven processes

typically execute in short bursts of processing separated by comparatively long periods of

idle waiting. The workflow framework must support long-running workflows by

persisting their state to a permanent data store and unloading them from memory when

they enter an idle state.

7

2.3.2 Transparency 

A system process often executes in one batch and yields one of two outcomes:

success or failure. In contrast, a human-driven process transitions among a series of

states until it ultimately reaches the acceptance state. The workflow may idle in a given

state for extended periods of time while it waits for human intervention. Businesses need

transparency, or visibility, into the state of the workflow. Indeed, one of the most critical

problems with Client Company’s existing manual process is the lack of transparency into

the state of the workflow.

2.3.3 Flexibility 

Whereas system processes are generally rigid and predefined, human-driven

processes tend to be more flexible and dynamic, due to the inherent differences in the

way machines and humans operate. In system workflow, the process dictates what the

participants (i.e., the software) must do, whereas in human workflow, the inverse is true;

the participants (i.e., the individuals) dictate what the process should do.

One way to achieve this flexibility is to replace the sequential model of execution

with one based on the concept of a state machine. A state machine is event-driven, and

does not have any predefined path of execution. For example, depending on an

approver’s decision, the AFE workflow could take totally different paths of execution.

When an AFE is approved, the workflow could transition from the “Awaiting Approval”

state to the “Approved” (acceptance) state. Conversely, when an AFE is declined, the

workflow could instead transition from the “Awaiting Approval” state to the “Awaiting

Revision” state. Upon revision and resubmission of the AFE, the workflow could then

transition back to the “Awaiting Approval” state. As the number of possible events

increases, the workflow quickly approaches an infinite number of potential paths of

execution. At the same time, designing the workflow using a sequential model of

execution becomes increasingly difficult.

Another way to increase flexibility is to allow the workflow to be modified at

runtime. New workflow instances can be created from a template, but diverge from the

8

template at runtime by adding, removing, and rearranging steps as necessary. For

example, AFE workflow instances could start with a default approval chain based on

business rules, but allow approvers to be skipped at runtime to accommodate exceptional

circumstances.

2.3.4 Maintainability 

A system process is typically data-driven; it receives some input, performs some

processing and auxiliary tasks, and returns some output. In contrast, human workflow is

often driven by a set of complex business rules. As businesses change, these rules can

change as well, requiring modification of the workflow.

Many human workflow systems have business rules implemented in code, which

is costly to maintain, as changes to business rules require code changes. Other human

systems have business rules defined in the application’s relational database, resulting in

overly complex database schemas. A robust human workflow framework should provide

dedicated storage and maintenance of business rules. The current trend is to use some

type of human-readable markup language supplemented by a visual rules editor.

2.3.5 Extensibility 

Human workflow systems often involve many of the same activities. For

instance, the submission-approval pattern exemplified by the AFE workflow is an

extremely common scenario. That said, each human-driven process has its own unique

characteristics. For this reason, it is insufficient for a workflow framework to simply

provide a library of common activities. These activities must be extensible by supporting

the addition of custom code.

2.3.6 Risk 

As with any software platform or library, the choice of workflow framework

involves some degree of risk. Many companies prefer to use a well-established product

with proven success in similar systems. On the other hand, a leading-edge workflow

9

framework may provide a superior solution, but at the cost of increased risk. Newer

technology is often perceived as immature, and sometimes undergoes drastic changes in

successive versions. Pre-release software in particular is often subject to feature changes

and application programming interface (API) refactoring.

2.3.7 Cost 

Workflow frameworks range from free, open-source libraries to custom-built,

million-dollar platforms, with varying degrees in between. Client Company has stated

that cost is unlikely to be the deciding factor, unless the discrepancy is exorbitant. For

the most part, a cheap, inferior solution should not be favoured over a more expensive,

superior one.

2.4 The Analytic Hierarchy Process 

2.4.1 Overview 

The subjective nature of the evaluation criteria makes it somewhat difficult to

provide a robust, objective analysis of the alternatives. In recognition of this challenge,

this report leverages MCDA, a management sciences methodology that uses simple

mathematics to analyze complex decisions [2]. More specifically, this report uses the

Analytic Hierarchy Process (AHP), an MCDA approach published by mathematician

Thomas Saaty in 1980 [3]. AHP transforms a set of qualitative comparisons into a set of

quantitative scores. A weighted objective function consolidates the scores for each

alternative into a single, useful metric. For the sake of brevity, the AHP calculations

themselves are not included in the body of the report, but are instead outlined in

Appendix A.

After establishing the criteria and alternatives, the next step in AHP is to define a

rating scale that converts qualitative comparisons into numerical values that can be used

in calculations; this scale is illustrated in Table 2.1.

10

Table 2.1: Rating Scale

Factor Subjective Rating

1 Equally preferable

2 Slightly preferable

3 Significantly preferable

4 Very preferable

5 Extremely preferable

2.4.2 Criteria Weighting 

The analyst uses a rating scale similar to the one above to quantify the relative

importance of the criteria. The result is a set of weightings for the criteria, as shown in

Table 2.2.

Table 2.2: Criteria Weighting

Criterion Weighting

Persistence 24%

Transparency 24%

Flexibility 24%

Maintainability 10%

Extensibility 6%

Risk 6%

Cost 6%

A full justification of these weightings is outside the scope of this report. Xyz has

rated the relative importance of the criteria based on discussions with Client Company.

11

2.5 The Alternatives 

2.5.1 Microsoft BizTalk Server 2006 

BizTalk Server 2006 is an enterprise server application that automates business

processes and integrates disparate software applications. BizTalk was originally

designed for EAI and B2B processes, but businesses are increasingly using it to automate

human workflow as well. The current version is the fourth release of the product, and is

an evolutionary update to the previous version, BizTalk Server 2004.

2.5.2 Microsoft Windows Workflow Foundation 

Windows Workflow Foundation (WF) is a component of Microsoft’s next-

generation developer platform, the .NET Framework 3.0, which will be included in

Windows Vista and ported to Windows XP and Windows Server 2003. WF’s goal is to

provide a unified framework for both system and human workflow. Microsoft is

targeting WF not only for third-party developers, but also for use in future versions of its

own products, including the 2007 Office System and the next version of BizTalk [4].

2.5.3 Alternatives Not Considered 

Although BizTalk was met with immediate success in the EAI and B2B market, it

was criticized for its shortcomings when used to automate human workflow. Microsoft

responded to this criticism by including Human Workflow Services (HWS) in the 2004

release. In brief, HWS was not well-received and did not see widespread adoption; as a

result, BizTalk’s workflow engine will be replaced by one built on WF for the next

release of the product, at which point HWS will become a deprecated technology. Since

HWS will eventually become unsupported, Xyz cannot consider its use for the AFE

project.

While this report considers two Microsoft technologies, there are a number of

third-party, enterprise-class workflow platforms available, such as SourceCode’s K2.net,

Captaris’ Workflow, and Ultimus’ BPM Suite. The problem with these alternatives is

12

that they are prohibitively expensive; the licensing for many of these platforms can cost

tens or even hundreds of thousands of dollars. Client Company is not prepared to make

such a large investment solely for the AFE project, and there are no immediate plans to

automate other business processes. While Client Company has not set a maximum cost

for the project, they have left consideration of technology investments to Xyz’s

discretion.

13

3 Evaluation 

The following subsections provide a direct comparison of BizTalk and WF

against each criterion. The AHP calculations transform the subjective ratings into

weighted, numerical scores.

3.1 Persistence 

Both BizTalk and WF provide a built-in persistence mechanism, and do so using a

very similar approach: workflow state is serialized and stored in a dedicated SQL Server

database. With respect to persistence, BizTalk and WF are equally preferable.

3.2 Transparency 

BizTalk excels in this area through a feature called Business Activity Monitoring

(BAM), which allows workflow milestones and key performance indicators (KPIs) to be

logged to a SQL Server database. BizTalk Server 2006 introduces several new features

that leverage BAM, including the BAM Portal, a web-based monitoring tool, and the

Alert Manager, which can subscribe to workflow events and automatically send out e-

mail alerts [5].

Like BizTalk, WF provides a tracking service that uses a SQL Server database;

however, WF provides only the tracking service itself, and does not implement any

delivery channels for it. The developer can use a reporting framework such as SQL

Server Reporting Services to provide the functionality of BizTalk’s BAM Portal, and a

custom notification service to provide the functionality of BizTalk’s Alert Manager.

While BizTalk and WF both provide a base framework for workflow tracking,

BizTalk provides built-in delivery channels for the information that a WF developer

would have to implement. For this reason, BizTalk is slightly preferable to WF.

14

3.3 Flexibility 

BizTalk’s origins as a systems integration platform are evident in its lack of

flexibility. This shortcoming is the main reason why its workflow engine is being

deprecated in favour of one built on WF in the next version of the product. BizTalk

workflows must be designed in a sequential fashion; there is no support for a state

machine model of execution. In addition, workflows must be strictly defined at design

time, and are immutable at runtime.

WF was designed from the start to provide flexibility for human workflow.

Workflow designers have a choice between a sequential and a state machine model of

execution. Furthermore, either type of workflow can be modified at runtime using an

API dedicated to dynamic modification. With respect to flexibility, WF is very

preferable to BizTalk.

3.4 Maintainability 

In BizTalk, a business rule is implemented as a condition and a set of actions, and

represented as an IF-THEN statement. The Business Rules Engine defines rules using a

declarative language called XLANG, an implementation of Extensible Markup Language

(XML), and stores them in a dedicated SQL Server database. The Business Rule

Composer serves as a visual rules editor. Having defined a set of rules, the workflow

designer can insert a Policy object to load and evaluate the rules at a specific point in the

business process.

WF’s Rules Engine is very similar to BizTalk’s Business Rules Engine. Like

BizTalk, WF represents business rules as IF-THEN statements; as a convenience, WF

developers can also include an ELSE clause. Rules are defined using Extensible

Application Markup Language (XAML), another XML-based markup language, and are

stored in a dedicated rules file. The RuleSet Editor and Policy object are very similar to

their counterparts in BizTalk. With respect to maintainability, BizTalk and WF are

equally preferable.

15

3.5 Extensibility 

While BizTalk provides a comprehensive library of common activities, these

activities are sealed, and cannot be supplemented with custom code. Similarly, although

the workflow as a whole is ultimately compiled into a class, there is no provision for the

developer to include her own custom code in this class. These limitations aside, it is

possible for the developer to invoke method calls into external assemblies through the use

of the Expression object and the associated Expression Editor, which allow the inclusion

of short code snippets.

WF also provides a library of common activities, but many of these activities can

serve as base classes for custom activities. In addition, WF uses a partial class model to

allow for easy addition of custom code. The workflow designer generates a markup file,

while the developer writes custom code in a separate file; the compiler then consolidates

the two files into a single class. The workflow designer can include a Code object to

invoke calls into the custom code at a specific point in the business process. With respect

to extensibility, WF is significantly preferable to BizTalk.

3.6 Risk 

BizTalk is an established product, in terms of both its role in Microsoft’s line of

offerings and its use by businesses worldwide. The 2006 version is the fourth release of

the product, and Microsoft has committed to at least two future versions. Furthermore,

customers can expect support for at least the next decade.

In contrast, WF is pre-release technology, and is currently in the Release

Candidate (RC) stage. There is no guarantee that Microsoft will not introduce breaking

API changes between now and the release date, or even that WF will be released at all.

The former is the primary concern, but the technology is fairly close to release, and the

fact that the last two RC versions have not introduced any breaking API changes is an

encouraging sign. The latter, although possible, is fairly inconceivable given not only the

16

late stage of the project, but also the fact that WF is a key component of the .NET

Framework 3.0, the next version of Microsoft’s developer framework.

Although the risk associated with WF seems fairly low, the fact remains that

BizTalk is an established product and WF is pre-release technology. As a result, BizTalk

is significantly preferable to WF.

3.7 Cost 

BizTalk Server 2006 Standard Edition is licensed at $8,500 (US) per processor.

Thus, Xyz has estimated the cost in Canadian currency and including all taxes to be

$10,846 for a single-processor server or $21,692 for a dual-processor server. The actual

price may vary depending on any existing licensing agreements between Client Company

and Microsoft, but the above figures provide an approximate estimate suitable for

comparison purposes.

In contrast, WF is a freely redistributable component of the .NET Framework 3.0,

and does not require payment of any licensing fees, even for use in a production

enterprise system. With respect to cost, WF is extremely preferable to BizTalk.

17

4 Results 

The criteria weightings and the subjective comparisons serve as the parameters to

the AHP calculations, whose results yield a set of weighted scores. Table 4.1

summarizes these weighted scores, while Figure 4.1 provides a graphical representation

of the same data.

Table 4.1: Weighted Scores

Criterion BizTalk WF

Persistence 12.2 12.2

Transparency 16.2 8.1

Flexibility 4.9 19.4

Maintainability 4.9 4.9

Extensibility 1.4 4.3

Risk 4.3 1.4

Cost 1.0 4.8

Total 44.8 55.2

18

0

10

20

30

40

50

60W

eigh

ted

Scor

e

BizTalk WFAlternatives

Multi-Criteria Decision Analysis

CostRiskExtensibilityMaintainabilityFlexibilityTransparencyPersistence

Figure 4.1: Graphical View of Results

The results show that BizTalk’s BAM Portal and Alert Manager resulted in a

significant advantage with respect to transparency, while its lower degree of risk as a

technology investment gave it an additional, minor advantage. Conversely, WF’s support

for state machine workflows and runtime modification resulted in a considerable

advantage with respect to flexibility, while its superior extensibility and substantially

lower cost gave it additional, minor advantages. Overall, the AHP calculations indicate

that WF is the superior solution, with a numerical advantage of just over ten percentage

points. It is also interesting to note that WF has a higher weighted score even when the

cost component is ignored.

19

5 Conclusions 

The results of the evaluation indicate that BizTalk and WF are very similar in

some aspects and very different in others. The large difference visible in some factors is

mainly a natural result of the history and design philosophies of the two alternatives.

WF’s state persistence mechanism and business rules framework are obvious

carryovers from BizTalk. With respect to persistence and maintainability, both

alternatives provide a robust solution, and they fare equally well.

Whereas WF is a freely distributable developer framework, BizTalk is a robust,

enterprise-class server application, and this distinction is evident in the tracking tools

provided. In addition to providing the base tracking service, BAM provides several

delivery channels for the information, including the BAM Portal and the Alert Manager.

The most influential factor on the outcome was flexibility for human-driven

business processes. BizTalk was intended for EAI and B2B workflow, and this is evident

in its lack of support for state machine workflows or runtime modification, both of which

are supported by WF.

Ultimately, WF’s superior flexibility for human workflow makes it the more

appropriate choice for the AFE system. It is also important to recognize that even with

cost ignored, WF is still the better solution.

20

6 Recommendations 

In light of the results of the analysis, this report recommends that the software

architect design a solution built on WF. The increased flexibility of the framework

makes it the more appropriate choice for a human workflow solution such as the AFE

system. Members of the development team should undergo whatever training is

necessary to familiarize themselves with WF. Previous experience with BizTalk may

also serve as an asset, as some of the concepts in WF are carryovers from BizTalk.

To compensate for the lack of some of the tracking tools that would otherwise be

provided by BizTalk’s BAM, the development team should provide transparency into the

workflow by leveraging a reporting framework such as SQL Server Reporting Services

and by creating a custom notification service. Neither of these is an exceptionally

complex task relative to the project as a whole.

There is a slightly higher risk associated with choosing WF rather than BizTalk,

mainly due to the possibility of breaking API changes and the need to eventually migrate

to the final release. The project manager should take these factors into consideration

when quoting time estimates and formulating a deployment plan.

21

References 

[1] D. Hasselback, “Client Company’s quarterly profit”, The Financial Post, Aug. x,

2006, [Online], Available: http://www.canada.com/nationalpost/

financialpost/story.html?id=e250af6e-b7c8-471c-819b-48742743411e&k=69095,

Accessed: Sept. 5, 2006.

[2] J. Dodgeson et al., Multi-Criteria Analysis Manual, tech. report, Dept.

Communities and Local Govt., UK Govt., 2000.

[3] J. McCaffrey, “Test Run: The Analytic Hierarchy Process”, MSDN Magazine,

June 2005, pp. 100-107.

[4] “Enabling People-Ready Processes through Business Process Management

(BPM)”, white paper, Microsoft Corp., June 2006.

[5] “Business Activity Monitoring”, white paper, Microsoft Corp., April 2005.

22

Acknowledgements 

I would like to thank Dr. Andrew Morton and Santheeban Kandasamy for

suggesting the use of MCDA for this report. They provided me with the background

necessary to perform my own research on MCDA.

I would also like to thank my colleague, Colleague, for his insight into the

application of BizTalk Server to human workflow systems. Mr. Colleague’s expertise on

BizTalk and his experience building similar systems proved invaluable.

23

Appendix A: The Analytic Hierarchy Process 

The Analytic Hierarchy Process (AHP) provides a robust, quantitative analysis of

alternatives when dealing with multiple, subjective criteria. It converts qualitative

comparisons into weighted scores, which represent each alternative’s total value using a

single, useful metric. AHP is based on binary comparisons (that is, comparisons between

two items at a time), since it is easier to compare a pair of items than to rank a large set of

items.

The first step in AHP is to define a rating scale that converts qualitative

comparisons into numerical values that can be used in calculations. The analyst can use a

similar rating scale to evaluate both the importance of the criteria and the quality of the

various alternatives. Table A.1 illustrates a typical rating scale:

Table A.1: Rating Scale

Factor Subjective Rating

1 Equally important/Equally preferable

2 Slightly more important/Slightly preferable

3 Significantly more important/Significantly preferable

4 Much more important/Very preferable

5 Extremely more important/Extremely preferable

The next step is to assess the relative importance of the criteria, with the ultimate

goal of computing a set of criteria weightings. The analyst prepares a matrix with the

criteria listed along both the vertical and horizontal axes, as shown in Table A.2. Each

cell then represents the importance of the criterion on the vertical axis relative to that of

the criterion on the horizontal axis, in accordance with the rating scale. A criterion is

obviously equally important as itself, so the cells along the diagonal contain unity. A

value less than unity indicates an inverse relationship (i.e., the criterion on the vertical

axis is less important than the one on the horizontal axis).

24

Table A.2: Criteria Matrix

Pers

isten

ce

Tran

spar

ency

Flexib

ility

Mai

ntai

nabi

lity Ex

tens

ibilit

y

Risk

Cos

t

Persistence 1 1 1 3 4 4 4 Transparency 1 1 1 3 4 4 4 Flexibility 1 1 1 3 4 4 4 Maintainability 1/3 1/3 1/3 1 2 2 2 Extensibility 1/4 1/4 1/4 1/2 1 1 1 Risk 1/4 1/4 1/4 1/2 1 1 1 Cost 1/4 1/4 1/4 1/2 1 1 1

The analyst then computes the sum of each column, and divides each cell in the column

by that sum, yielding a normalized matrix, such as the one shown in Table A.3. To

account for any inconsistencies, the analyst computes the average of each row of the

normalized matrix; each row’s average represents the percentage weighting for the

criterion associated with that row.

Table A.3: Normalized Criteria Matrix

Pers

isten

ce

Tran

spar

ency

Flexib

ility

Mai

ntai

nabi

lity Ex

tens

ibilit

y

Risk

Cos

t

AV

ERA

GE

Persistence 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Transparency 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Flexibility 0.244 0.244 0.244 0.260 0.235 0.235 0.235 0.243 Maintainability 0.0816 0.0816 0.0816 0.0869 0.117 0.117 0.117 0.0978 Extensibility 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576 Risk 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576 Cost 0.0612 0.0612 0.0612 0.0434 0.0588 0.0588 0.0588 0.0576

The analyst uses a similar process to perform the actual evaluation. For each

criterion, she creates a comparison matrix to compare the alternatives against that

criterion. The analyst then normalizes the matrix and computes the average of each row,

which in this case represents each alternative’s score for that criterion. To obtain the

weighted scores, she then multiplies each alternative’s score by the criterion’s weighting.

25

Tables A.4 and A.5 show comparison matrices for the ‘flexibility’ criterion in initial and

normalized form, respectively.

Table A.4: Comparison Matrix

BizTalk WF

BizTalk 1 1/4

WF 4 1

Table A.5: Normalized Comparison Matrix

BizTalk WF AVERAGE WEIGHTED

SCORE

BizTalk 0.2 0.2 0.2 0.049

WF 0.8 0.8 0.8 0.194

After computing the weighted scores for all of the criteria, the analyst computes

the sum of the weighted scores for each alternative, yielding a set of total scores. The

analyst often multiplies these scores by a scale factor; for example, to use a 100-point

scale, she multiplies the values by 100. Table A.6 illustrates the computation of the final

scores.

Table A.6: Final Scores

Criterion BizTalk WF

Persistence 0.122 0.122

Transparency 0.162 0.081

Flexibility 0.049 0.194

Maintainability 0.049 0.049

Extensibility 0.014 0.043

Risk 0.043 0.014

Cost 0.010 0.048

Total × 100 44.8 55.2