Editor's Notes

While teaching freshman composition years ago, I stumbled onto a paradox: The more requirements I specified for an assignment, the more creative were the essays my students produced. Why was this so? I think because requirements define particular problems for us, and we are by nature problem solvers. The command to "Be creative" without any set guidelines induces panic and scrambles thinking, but "Solve this problem" sets minds to wrapping around the challenge and sparking creative solutions.

This is a valuable thought to hold as you read this month's collection of requirements-focused articles: Jim Heumann on the importance of taking time to gather requirements at a project's inception; Peter Eeles' practical advice and templates for specifying architectural requirements; Ellen Gottesdiener on her "Wall of Wonder," a collaborative storyboarding technique for generating requirements and other project deliverables;* Ben Lieberman on leveraging requirements to manage project scope, and Brenda Cammarano on the powerful integration between Rational RequisitePro and Rational Rose. To all the compelling business and technical reasons these article present for compiling formal requirements, we can add the argument that requirements provide a fundamental structural prerequisite for creativity -- pretty powerful stuff for a software development team.

There's more thought-provoking material in this issue, too. In Franklin's Kite, Joe Marasco takes an unorthodox look at assessing project progress from a physicist's perspective. In our Features section, Murray Cantor introduces the new Rational Unified Process for System Engineering, a RUP derivative that's now available through Rational Services. The article by Gilles Pitette in the Management section explores the synergy between RUP and Progressive Acquisition, a widely-used, iterative approach to large-system procurement. Tom Milligan explains how to use Rational ClearCase for PERL files in the Tech section, and our two book reviews preview, respectively, Noel Tichy's The Leadership Engine, and a new volume on Programming Web Services with XML-RPC from O'Reilly.

Here's to happy reading and a Happy Thanksgiving -- and as you head toward the twin demands of holiday shopping and end-of-quarter goals, may all your requirements be well specified!

Entire issue in .pdf

Download the entire issue in .pdf(2.14 Mb)

Marlene EllinSenior Editor

*Note that our cover graphic (which also appears in Ellen Gottesdiener's article) this month is a wall painting from the Cave of Chauvet in Vallon-Pont-d'Arc, France. Anthropologists believe these expressive images represent an early form of collaborative requirements planning -- that helped the group of Aurignacians who dwelt in the cave prepare for a successful hunt.

Copyright Rational Software 2001 | Privacy/Legal Information

What Does "No Time for Requirements" Mean?

by Jim Heumann Requirements Management

EvangelistRational Software

Have you ever heard someone say something like the following?

"We don't have time to gather requirements. If we don't start coding right now, we'll never meet our deadline."

Such statements -- which are not uncommon -- represent important clues about the culture and maturity of an individual or a team. When someone says there is no time for requirements, what does that really mean, and what are the implications? Let's take a closer look.

The term requirement has many definitions; each method, process, methodologist, and pundit seems to have a unique twist on what a requirement actually is. All would probably agree, however, that requirements should clearly state what it is that a given software project is supposed to build and deliver. If we substitute this simple definition for the term requirements in the statement above, then it reads like this:

"We don't have time to figure out what we need to build and deliver. If we don't start coding now…"

Seems unlikely that someone would make that statement out loud, doesn't it? So what is really going on when people say they don't have time to do requirements? And what happens if the requirements don't get done?

What Does It Mean?

I believe that when people make the statement in question, what they really mean is one of the following:

1. "I already know the requirements -- or I can figure them out on the fly."

2. "The requirement specifications I've worked with in the past have been so bad -- I think I'm better off without such things."

3. "I don't really care what we build, as long as we get it done on time (and collect our money)."

4. "This application idea is so new (or so unique) that we just can't pin down requirements in advance."

Each possible interpretation has its own perils, as we shall see.

"I Already Know…"

This is the "kindest" interpretation. The people (or person) in charge of the project feel that they implicitly know what the requirements are or that they can figure them out as they go, while coding. If we dig a little deeper, though, we see that what they really mean is, "My developers know the requirements or can figure them out." In the end, developers are the ones who implement the functionality of any software program; if the specifications about what to build (i.e., the requirements) are not written down and agreed upon, then it is developers who will ultimately make the call on what gets implemented.

What's wrong with that? In many organizations, that is pretty much how they have always done it. Well, one reason why it is no longer acceptable to do things this so-called "old-fashioned way" is that projects today are more complex than they used to be. And they are more complex in two domains: the problem domain and the implementation domain.

To begin with, software has penetrated areas of our lives that we never even imagined it would affect ten years ago: We have software for managing bull breeding, navigating automobiles, detecting fraud in cellular phone use, and for many, many other specialized purposes; most developers cannot possibly know or understand the problems to be solved in these particular areas. And as application areas have grown more complex, so has programming. Distribution, concurrency, real-time, interrelated components, new languages, and other issues that developers must consider in order to create a robust system, all place pressure on them to become more and more expert at writing code. Also, these new demands mean that developers have even less time and capacity for figuring out the problem the system needs to solve.

Another reason not to rely on developers to "figure it out on the fly" is that they have an inherent conflict of interest. They have deadlines, sometimes based on their own estimates, sometimes on others'. If you give them a very vague statement of what to do, they will have many choices about how to implement the functionality. It would be natural for them to choose the one that most closely matches their main goal -- which is usually

meeting the deadline.

I once had the job of writing additional code for a GUI (Graphical User Interface) builder that had no facility for the user to specify the color of elements such as push buttons, list boxes, and so on. The stated requirement for the functionality I had to add was "Allow the user to specify color for GUI elements." Now, there are a lot of ways you can allow users to specify a color: You can give them a list of colors in text format; you can present a bunch of boxes in various colors and have them choose; you can show them a color wheel and ask them make a choice; you can ask them to type in numbers between 1 and 256 for red, green, and blue (RGB) values, then combine those to make any color they like. Some options make it easy for the user and harder for the developer; some reverse that relationship.

I am chagrinned to admit it now, but I chose to have the user type in RGB values. Why? Because that was the easiest option for me to implement, and it allowed me to stay on schedule. The problem was, it was the hardest option for a user. Why do I mention this embarrassing example? Because it shows that forcing developers to make important decisions about user requirements can put those developers in untenable situations and often leads to unsatisfactory results.

"I've Had Past Experiences with Bad Requirements"

Another reason managers may think that gathering requirements is not worthwhile is that they have run up against bad requirements in past projects. Bad requirements come in many forms, but the bottom line is that they either don't provide enough information to really build anything or they provide information that is misleading or contradictory. In such cases, it is true that the project would be better off if everyone just made their best guesses. But remember: Just because you had to work with bad requirements in the past doesn't mean it always has to be that way. Helpful guidelines, techniques, and tools are available to help your team create good and useful requirements.

Good and Bad RequirementsAccording to the IEEE 830 Documentation Standard for a Software Requirements Specification, the criteria for a high-quality requirements set are as follows:

Unambiguous -- Every statement has one interpretation. Terms are clear and well defined. Complete -- All significant requirements are included. No items have been left for future definition. Verifiable -- All features specified as part of the project have a corresponding test to determine whether they have been successfully delivered. Consistent -- Conflicting terminology, contradictory required actions, and impossible combinations are notably absent. Modifiable -- Redundancy is absent; index and contents are correct. Traceable -- Each referenced requirement is uniquely identified. Correct -- Every stated requirement represents something required of the system to be built. This may sound obvious, but it is surprisingly easy to

include extraneous requirements, or requirements that really pertain to a separate system entirely.

How do you recognize bad requirements? Compare them against the criteria above; if they don't match, then beware.

I recently came across the following requirement in an actual project requirements document I was reviewing:

"The application must be extremely fast and powerful."

Clearly, this is a poor requirement because "fast" and "powerful" are highly subjective terms, so there is no way to verify whether the system meets the requirement.

Organization and Tools Can HelpIn some instances, requirements are not intrinsically bad, but they are unusable because there are so many of them -- and they may be poorly organized. Traditional software requirements specifications often contain thousands of declarative statements that "The system shall…" do something. It is very hard for anybody to understand several thousand things at once. Classifying and organizing the requirements can help. Splitting up the big list into smaller lists in a hierarchy that identifies explicit relationships represents a big step forward in understandability and usability. Writing the functional requirements as a user-oriented "story" can also help. Use cases provide a mechanism to highlight how a given "actor" uses the system and what the system does in response. This goal-oriented approach puts the requirements in context and helps to make them easier to understand.

Tools can help make requirements usable, too. Having the requirements in a specialized requirements database rather than in a collection of unrelated documents makes them much easier to organize, search, and sort. People filling different roles in a software development team can "slice and dice" them any way they choose. A project manager, for example, can use the requirements to help keep the project on track, using the tool to find information about how many requirements have been implemented, how many new ones have been added, and how many have changed during the last week. An architect can search for all of the requirements that are specified as difficult to program and that affect the architecture, so that these can be implemented early in the project in an effort to drive out technical risk. Tools won't help you write good requirements, but they can help you organize them and find the information you need.

"I Don't Really Care"

This is the saddest case. Some people "don't have time" for requirements because they don't really care if they build the right thing. Perhaps they are working on a project that they feel is doomed due to political or organizational problems, so they think that doing good requirements won't help anyway. Or perhaps they have chosen to work on a given project for personal reasons, maybe to get experience in a new technology or

technique. Whatever the reason, if people don't care, then you have a problem that organization and tools can't solve.

"This Application Is So New…"

Software projects aren't always trying to solve well-defined problems with known solutions. Many seek to expand the boundaries of what is possible for computers. And if there is no precedent for your system, it is very hard to get potential users to tell you how they want it to work. In 1978, Dan Bricklin developed VisiCalc, the first electronic spreadsheet, on an Apple ][ while attending Harvard Business School. There was nothing like it at the time; it is unlikely that he could have gotten requirements from "users," simply because there weren't any. Similarly, consider Napster. It is also unlikely that Shawn Fanning, the 18-year-old Northwestern University dropout who developed Napster in 1999, did extensive requirements gathering before working eighteen hours a day for three months to write the original source code and deploy the site -- yet it was a huge success.

It is sometimes said that such systems are "exploratory software engineering." For their first release, extensive requirements gathering is unlikely to add much value. Subsequent releases, however, are a different story. Once users catch on, they develop certain wants and expectations; developing new features that you think are "cool" may not meet those expectations. Asking users what they need, expressing it clearly, writing it down, organizing it, and using it to design and implement new functionality can help ensure that a good idea grows instead of fizzles.

Lifecycle Implications

When people say they don't have time to do requirements, they are usually thinking only about the first release deadline and not about the implications that omission may have for other concerns throughout the project lifecycle, including schedule, architecture and design, test and performance, and usability issues.

Schedule

If you don't know what you are supposed to do, how do you know when you are done? A prevalent problem for many software projects is late delivery, often caused by "feature creep." Not having control of what is supposed to go into a given release opens the floodgates to ad hoc requests for more features to develop and implement. Having no means to understand the relative value of each request and make informed decisions can easily lead to "biting off more than you can chew" and missing deadlines. With well-defined requirements, you can more easily turn down requests for features that do not further your goals and maintain tighter control of project scope.

Architecture and Design

Without a high-level understanding of the range of functionality that must go into a system, it is easy to create a design that is not adaptable or scaleable. An architecture that meets existing requirements but does not

take future changes into account may end up being hard to modify or extend. Well-defined requirements anticipate modifications and changes, providing a framework for a flexible architecture and overall design.

Test and Performance

If you test software without requirements, then you are merely "making sure the application does what it does" without crashing. Testers can look for GUI elements that don't work, system crashes, and more obvious problems, but in the end they have no means to verify that the program really meets the user's objectives. One simple definition of quality software is that "it does what it is supposed to do." Requirements specify what a system is supposed to do, so from very early on in the development lifecycle, you can begin testing to ensure that you are actually creating the workable system your user needs.

Usability

Even if you have functional requirements to design and test against, failures can still occur. If the team doesn't know, for example, how simple the user interface and how fast transaction times must be, how many users the system must support, and who has to maintain the system, then you can easily wind up with an application that provides all of the requested functions but does not solve the problem. By providing information about non-functional requirements, you can ensure that your system designers and developers won't spend all of their time implementing the functionality without considering other factors that are vital to the system's success.

Dig a Little Deeper

So now, if you hear people in your organization saying, "We don't have time for requirements," then consider the implications. If your team is inventing something brand new, then you might be able to get away without gathering requirements. But if, like the majority of organizations, you are working in domains for which models and precedents are plentiful, then it may be worth your while to dig a little deeper and find out what those people really mean. Do they intend to let developers make important decisions outside their respective areas of expertise? Do they understand the difference between a good requirement and a bad one? Do they simply not care what system you develop? Do they understand the potential impact on other areas of the project lifecycle if they don't specify requirements? If you understand what is really going on below the surface, you can respond to such statements in a way that can help make your project rather than break it.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

RUP SE: The Rational Unified Process for Systems Engineering

by Murray Cantor Principal Consultant

Rational Software Corporation

Systems engineering differs from software engineering in at least one fundamental respect: It addresses a broader set of requirements than a software-only effort would typically entail. This is because, in addition to its software components, a system also includes hardware and worker roles -- along with their attributes and the diverse relationships among them. Rational is currently working to develop RUP SE, a derivative of the Rational Unified Process® that specifically addresses the needs of systems engineering projects. Presently, a prototype is available through the Rational Services Organization as a deployment service package. By bringing the discipline and best practices of the Rational Unified Process to the challenges of system specification, analysis, design, and development, RUP SE represents an important advance in systems engineering that can help organizations save time, cut costs, reduce risk and improve the quality of the systems they build.

This is the first installment of a two-part introduction to RUP SE. Here I explain where RUP SE follows the RUP and where it departs. I'll also begin explaining the most significant new constructs in RUP SE; Part II will complete this explanation.

Why We Need RUP SE

The classic definition of a system is an assemblage of hardware, software, and worker roles that collectively meets a business purpose. Many software developers are de facto system developers, because nearly all software tools -- such as e-business applications, enterprise data

integration initiatives, and embedded software -- contain some elements of a systems problem.

But systems engineering isn't just software engineering with extra requirements. A systems engineering development process requires a set of activities to help you define the system architectural elements and their requirements. Moreover, systems design requires you to specify all these elements simultaneously. Systems developers may, for example, want to specify a range of system configurations, with similar architectures but different hardware or software components to meet various requirements like cost/performance tradeoffs. Hence the need for a RUP derivative that provides the guidance and templates necessary to support these systems-related activities, while conferring all the business and engineering benefits associated with the RUP.

In addition to pure systems engineering projects, RUP SE addresses the needs of projects that:

● Entail concurrent hardware and software development.

● Are sufficiently large to require more than one development team.

● Present architecturally significant deployment issues, such as a need for heterogeneous hosting platforms or the use of embedded processors.

As a RUP derivative, RUP SE helps to unify the entire system design and development team, and enhances communication and collaboration among team members. It does this by functioning as a comprehensive knowledge base, complete with a common modeling language and a coherent view of how to create a system that meets requirements. Teams and organizations can readily integrate RUP SE with their current procedures and best practices, because it retains the RUP's adaptability and ease of customization. A point-and-click, browser-based interface serves as a convenient access point for the guidelines, templates, and examples that embody both processes.

RUP SE 1.0: Available Through Rational Services

Rational is offering the first release of RUP SE as a deployment service package that combines classroom training, tool environment customization, on-site workshops, mentoring, and support.

Like the RUP, RUP SE is a customizable design and development methodology

RUP SE Extends the RUP

As a derivative of the RUP, RUP SE retains the RUP's cornerstone principles, which have been refined and extended to enhance their utility for systems engineering efforts:

● RUP SE brings concurrent design and iterative development to systems engineering. In addition, it provides the highly configurable workflow templates required to identify the hardware, software, and worker role components that comprise a systems engineering project.

● RUP and RUP SE both are designed to

delivered as a browser-based, Web-enabled product. (Along with Philippe Kruchten, I like to think of it as a process framework.) It includes guidelines, templates, documentation, and examples for critical systems engineering activities, representing the distillation of many successful systems development projects and ideas.

Our reasons for offering this first release of RUP SE as a deployment service package are two-fold: (1) to give early adopters the guidance and tool support necessary to get optimal results, and (2) to obtain the detailed feedback we need to refine and develop future product releases.

Like all Rational products, this one will be built on solid user experience and tailored to meet real needs.

help teams systematically define, organize, communicate, and manage requirements. Both inherently support change control and quality initiatives. Without these capabilities, no systems engineering project is likely to be deemed successful relative to cost or business objectives.

● RUP SE mirrors RUP in fostering a modular, component-based architecture. System architectures frequently involve a wider range of component types than software-only architectures. But both benefit from methodical, systematic design, development, and validation.

● RUP SE retains the RUP's emphasis on visual modeling via the graphical Unified Modeling Language (UML). UML provides a blueprint for visualizing, specifying, constructing and documenting system processes and functions, and even concrete artifacts like hardware components, programming language classes, and database schemas. RUP SE includes a set of UML artifacts for system architecture and specification.

For general background on the RUP, check the References section below.

Key Innovations in RUP SE

As you might expect, extending the RUP to systems requires additional activities, artifacts, and workflows.1 Here we will describe them briefly, and then discuss the important concept of locality. Part II of this article, which will appear in the December issue of The Rational Edge, will further elaborate on some of the important distinctions that we identify here.

● New Roles. In RUP SE, the development team includes system engineers, in addition to worker roles like architects, developers, testers, etc. As you might expect, the system engineer role is primarily concerned with the design and specification of the hardware and system deployment, to help address overall system requirements.

● New Artifacts and Workflows. RUP includes full support for software system concerns like usability, maintainability, performance, and scalability. RUP SE adds artifacts and workflows that address additional concerns in the systems engineering domain, such as security, training, and logistics support.

● An Emphasis on Business Modeling. Whatever kind of system you are architecting, it is important to understand the business purpose it will serve. Otherwise, system requirements will not accurately reflect business activities. RUP SE does not include changes to the business modeling features of RUP. However, RUP SE users are strongly encouraged to create business use cases with the associated identification of business actors and the flow of business events, in order to adequately define system requirements. Further, the RUP SE use-case flowdown activity is applied to derive system requirements from business requirements.

● Viewpoints for Systems Engineering. Both RUP and RUP SE define system architectures across two dimensions: viewpoints (i.e., areas of concern) and levels of specification (as represented by UML models). RUP SE supports domain-specific viewpoints common to system architectures, such as safety, security, and mechanical. Modeling levels are similar for most systems regardless of their complexity.

● Scalability Enhancements. Once design decisions have been captured in viewpoints and specified via model levels, the system architecture is captured in a set of UML diagrams; these further describe it from the various viewpoints and model levels. Although many of these artifacts are similar across RUP and RUP SE, there are a couple of important differences. In a nutshell, these new artifacts allow you to break the system down (1) by subsystems, and (2) by the localities where processing takes place. Each locality and subsystem has its own derived requirements in RUP SE, enabling the process to scale to meet the needs of even the largest and most complex projects.

● Allocated Versus Derived Requirements. In parallel with UML and RUP, RUP SE encompasses two types of system requirements: use cases, which capture functional requirements; and supplementary requirements, which cover non-functional elements like reliability and capacity. With respect to the requirements associated with subsystems and localities, RUP SE makes a further distinction between those requirements that are allocated and those that are derived. A locality or subsystem requirement is allocated if a locality or subsystem is assigned sole responsibility for fulfilling a system requirement. A locality or subsystem requirement is derived if it is identified by studying how the subsystem or locality collaborates with others to meet a system requirement.

● Subsystem Level Flowdown Activity. RUP SE derives system requirements from business requirements via use-case flowdown activities. However, RUP SE departs from the RUP in that it also specifies a flow of events in a subsystem level, "white box" view that references specific architectural elements. This extra step is necessary in order to make decisions about where events are hosted, and to relate processes to events.

● Support for Designing Additional Components. The design level specification of system components with RUP SE is similar to its software-only counterpart in RUP. The key difference, as previously mentioned, is that systems engineering typically entails

more different kinds of components than software engineering, such as hardware. Delineation of these components is supported via analysis of the RUP SE subsystem and locality use-case surveys that are generated prior to specifying component designs.

The Concept of Locality

RUP SE captures a system architecture via a set of diagrams, each of which expresses the architecture from a specific viewpoint and model level. The majority of these are standard UML diagrams. For example, an Analysis level model seen from the Computational viewpoint would describe the system in UML as subsystems that collaborate to meet user requirements. These subsystems would be further broken down into a detailed subsystem class model, which would comprise the Design level from the Computational viewpoint.

When it comes to the Engineering viewpoint, however, RUP SE extends the standard UML notation to include a special diagram at the Analysis level. To adequately support systems, the Engineering viewpoint must support the decomposition of systems into the elements that host the processing; we call these localities. And locality diagrams (more below) are the most abstract expression of this level of decomposition.

Localities describe the computing and storage resources where processing occurs in the system. However, at the Analysis level we do not specify a geographic location or even a kind of hardware relative to a locality. (That level of detail is captured in the Design level model, as you might expect.) For example, a Locality view might show that the system in question enables processing on a space satellite, as well as on a ground station. The processing hosted at each of these two localities would then constitute an important design decision.

Locality Diagrams

Locality diagrams express the initial distribution of system processing elements by locality, and the connections between them. For many systems engineers, the locality diagram constitutes "the architecture" of the system.

Locality diagrams consist of two elements:

● The localities themselves

● Connections, which specify the paths data can take between the localities

As illustrated in Figure 1, the elements of the diagram can be usefully represented as UML nodes; their icon is a rounded cube.

Figure 1: A RUP SE Locality Diagram

Figure 1 is a locality diagram that documents an engineering approach to a click-and-mortar e-business platform. The enterprise has a number of retail stores, central warehouses, and a Web presence. This approach allocates processing capability to the stores.

Coming in Part II: Requirements Analysis and Component Level Requirements Specification

As stated above, a locality is an element of the system that hosts processing -- but what processing does it host? Each locality can provide or host a set of logical subsystem use cases, which are derived from a use-case flowdown activity. This helps partition system design efforts among multiple teams, thus ensuring that RUP SE scales to accommodate even the biggest projects.

Next month, in Part II of this article, I'll talk more about requirements analysis in RUP SE, from the point of view of subsystem use cases and how to derive them. I'll also explain how you perform component level requirements specification in RUP SE. In conclusion, I'll discuss the key management and organizational concerns for system development that RUP SE helps address.

Notes

1 Some quick definitions: activities are the tasks that must be performed to create the system; that is, they describe "how" to proceed. Artifacts are the deliverables; they describe what is to be built. Workflows document and manage lifecycle steps, which specify "when" events happen relative to one another. A worker is a role -- not an individual -- that describes "who" performs an activity.

References

Philippe Kruchten, "What Is the Rational Unified Process?" The Rational Edge, January 2001, http://www.therationaledge.com/content/jan_01/f_rup_pk.html

Philippe Kruchten, The Rational Unified Process -- An Introduction, 2nd ed., Addison-Wesley, 2000.

Rational Software's Rational Unified Process product page: http://www.rational.com/products/rup/index.jsp

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Specifying Requirements With a Wall of Wonder

by Ellen GottesdienerPrincipal, EBG Consulting

Most of us are familiar with the tension between some developers' desire for loose, flexible processes and project and business demands for rigorous, disciplined software development. In my experience, successful companies are those that assemble collaborative teams of dedicated software professionals who exploit the strengths of both positions. These people recognize that nuggets of rigor and discipline lie at the heart of well-executed agile methods, and that within the soul of disciplined methods lie adaptability and open-mindedness. In project after project, I have seen those with a collaborative vision and approach succeed where traditional, "lone wolf" approaches have failed.

Collaborate comes from the Latin words laborare (to work) and com (with), and so literally means to work together. Collaboration occurs when a group of people with a common and well-defined goal integrate their individual knowledge and skills to deliver on that goal. When I facilitate for development teams, I conduct collaborative workshops in which we create requirements specifications and related deliverables such as use cases, business rules, data and class models, and plans. The workshops comprise a healthy mix of business people (users and customers) and IT (information technology) professionals, all driven by a single goal: to deliver the best software solution to meet a business need.1 The processes and techniques we use in these workshops contribute to building healthy team relationships, trust, and shared meaning.

In this article, I describe one very effective collaborative technique -- the Wall of Wonder (WoW) -- which helps software teams produce the kind of detailed, sharply-defined requirements that effectively guide development. As an "emergent" deliverable, requirements evolve through exploration and examination using representative forms such as models and

prototypes. A collaborative approach allows business and IT specialists to explore their requirements through these means, while accommodating the necessary fluidity of the requirements process.

Using the Wall of Wonder

Collaboration requires shared space in which the deliverables can emerge, and the simplest -- and cheapest -- form of shared space is a wall. Walls have been used by generations of human societies to record events and tell stories (this spans the entire range of human history, from cave paintings to modern graffiti), and to express hopes and wishes. The Wall of Wonder (WoW) is both a communication medium and a storage place for the tools and materials (which we'll discuss below) and the content of the collaborative effort.

To make best use of the wall, groups need a structured process -- a fast, flexible way to leverage the wall space and engage in healthy interaction. Dynamics for healthy groups iterate around a well-recognized cycle first described by B.W. Tuckman,2 who labeled the phases forming, storming, norming, and performing. Forming involves finding common goals and purpose. During the storming phase, members engage in open disagreement, which, under healthy circumstances, strengthens the group and promotes deeper understanding. Norming is the process of finding commonly acceptable ways for all group members to interact. In the performing phase, the group becomes task-oriented and focuses on producing the agreed-upon deliverables. Although you can't take shortcuts within this nonlinear set of dynamics if you want usable results, you can at least accelerate it. Certainly, a group that has worked together previously, and whose members are familiar with each other before the WoW session, may begin norming and performing right away. To produce a deliverable, however, they will still need to follow all the steps described below.

Storyboarding is one kind of wall work that follows the cycle Tuckman describes. A storyboard is a series of continuing panels, sketches, or scenes depicting a plot or sequence of actions. In business, storyboards are popular for solving problems and creating collaborative plans. My Wall of Wonder approach is based on a storyboarding technique I learned from the Institute of Cultural Affairs.3 The group uses text or diagrams to build requirements, an iteration plan, or other important deliverables by successively using individual, subgroup, and whole group activities to generate items such as desired project features, business rules, use cases, data elements, screen navigations, and so on. I call it a Wall of Wonder (WoW) because of the wondrous results groups can achieve. Table 1 outlines the rationale for this approach, and Figure 1 shows a step-by-step flow.4 For a more detailed explanation of the steps, see the Sidebar.

Table 1. Collaboration Pattern: Wall of Wonder (WoW)

Name Wall Of Wonder (WoW)

Context

A group of stakeholders needs to solve a problem or create a deliverable such as a model or plan. A same time/same place group meeting seems to be a good choice because it will speed the process.

Problem

Unstructured group meetings are not always the best way for participants to interact. If there are introverts, more vocal individuals, or more organizationally powerful people present, some valuable input may be lost.

Solution

● In a group meeting, allow time for individual thinking, followed by small group work to combine individual ideas and then whole group work on the wall.

● Begin with a clear focus question.

● Have all work visible on the wall.

● Permit the group to logically arrange the elements on the wall

● Respect individual time and the need to think alone and then post the result on the wall.

● Establish a pattern of individual -> triad -> whole group -> individual.

● For longer collaborative events, rotate membership in subgroups.

Consequences

Integrating multiple perspectives produces a higher-quality product than one produced by a subset of individuals. Greater team collaboration and goodwill are established because the process allows participants to think both alone and together. As a result, teams can create complex requirements, models, plans, or structures in a short time (one hour to one day).

Entry Criteria

● Knowledgeable participants representing all key IT and business perspectives who are sharing the same time and place

● Understanding of what deliverables are to be created and what is required to create them

● Focus questions

● Room(s) with sufficient wall space

● Low-tech tools such as post-its or sticky wall with large cards; or high-tech tool such as an outliner or drawing tool along with a technographer

Exit Criteria

All deliverables are visible on one or more walls; the deliverables contain both detailed and summary items, are logically cohesive, and are understood and agreed upon by all participants.

Uses

Define use cases by release, determine the scope of events or use cases, define a data or class model, create an actor (role) map, specify business rules in a template format, associate business rules with use cases, define goals and objectives, design an organization, create a communication plan, generate a project, phase, or iteration plan, define selection criteria for a software package, design a low-tech user interface prototype, specify use case steps.

Figure 1: Wall of Wonder Process

At the beginning of each WoW activity, a neutral facilitator poses focus questions to trigger responses from participants. (The focus questions vary, depending on the purpose; we'll look at examples later). After people generate detailed items on their own, subgroups write agreed-upon items on detail cards or sticky notes. Here's where the wall comes into play. These cards are then displayed on the wall for the whole group to focus on. When everyone reaches agreement on the posted items, the cards are grouped under header cards (usually in a contrasting color from the detail cards). At the end of the process, the work of the whole group is displayed on a single, large wall.

Three Wall Strategies

In the bottom-up approach shown in Figure 1, participants begin with detailed items, and then work their way "up" to higher-level categories, designated with header cards. Along the way, they sort and eliminate items, including project features, business rules, use case steps, and data elements.

Steps for a WoW Session

1. Make individual listings.a) Briefly explain the overall flow to prepare participants. Make sure they have paper or cards, and then ask the focus question. (Remember, the focus question sets the content for the Wall of Wonder and is critical to the process). b) Ask the focus question. Allow people to think and brainstorm alone for five to seven minutes, listing their answers on the card or paper. c) Ask participants to mark their top three items, thereby sorting those they consider most important, relevant, useful, or critical. I prefer to give them a neutral criterion for sorting by saying, "Draw a star next to your top three items."

2. Assign one item per card, in subgroups.Divide the group into subgroups. Determine the subgroups ahead of time or form them at this point. The task of the subgroups is to arrive at a list of items in response to the focus question by sharing, merging, converging, and adding to their cards. Shoot for seven to nine items, creating a card for each. I usually have the group choose a leader to keep them on task and on time and a recorder to do the recording for the group. I instruct the recorders to:

❍ Use a black marker to write one item per card.

❍ Describe the item with three to five words (cut to the chase).

❍ Write big.

3. Post cards on the wall.Reconvene as a whole group. Ask for one card per group using questions such as "Which card was most obvious to your subgroup?" "Which card did you select the quickest?" or "Which is your most controversial card?" As you place each card on the wall, read it exactly as it is written

In a top-down approach, participants prepare the header cards before the workshop or in an earlier collaborative activity. Starting with these broad categories, participants then work their way "down" by supplying lower-level details. This approach is useful when the categories are well known and accepted by participants ahead of time. Examples of high-level categories or header cards might be project phases, SWOT (strength, weaknesses, opportunities, threats) analysis categories, use-case names, business rule sets, named data entities or business objects, or object lifecycle phases.

In a middle-out approach, participants generate items more randomly, without regard to categories. Then team members sort the items, specify more detailed items, and converge on the categories. An example might be asking participants to list use cases. After team members sort, combine, add, and clarify the cards, something that originally was posted as a use case may turn out to be a use-case step, a set of use cases, or a use case that is out of scope.

Group Size and Balanced Participation

At a WoW session, your goal is to foster healthy group participation, achieving a state in which, as Aristotle put it, "The whole is more than the sum of its parts; the part is more than a fraction of the whole." There is a delicate balance between group size and getting the right participants ready to work the wall.

When a group includes people with different knowledge, perspectives, and backgrounds -- as is often the case in requirements workshops - then communication is complex. That is one reason subgroups are useful. You'll want to form subgroups if you

and ask the group if they have clarifying questions. If there are any questions or puzzled looks, ask the team members who wrote the card to explain its meaning and intent. Continue this process until all detail cards are posted.

4. Group cards.As this process unfolds, guide the team in grouping the cards. (Often, participants intuitively start to tell you to do this. If they don't, ask them whether anything on the wall "goes together.") Moving the cards, form columns or clusters, depending on the deliverable. Create new cards whenever a new item arises, when an item needs to be clarified, or when header cards are suggested.

5. Create header cards.Create a header card for all groups or clusters. If you were writing a header card for data elements in a data model, for example, the header card would be the name of the data entity or business object to which the data element belongs. If you already have header cards from an earlier step, then ask participants to reconsider the label and ensure that it is still relevant and meaningful.

6. Analyze groupings. a) Extend the content of the wall by asking everyone to review his or her original list. As they do this, ask them whether any cards should be added. b) Quality check the end product (see Table 2 for examples), and then use your decision rule process to reach closure.5

Note: As a follow-up to creating the Wall of Wonder, consider conducting a structured discussion of the wall's contents. Ask the group to reflect on both the product they created (and its implications for the project) and on the process they used to create it. Pausing to debrief -- to process both your product and your process - offers rich learning opportunities and is the mark of a truly collaborative team.

have more than four participants. This is critical for preparing for the "plenary" part of the process (Steps 3-6 in Figure 1). In later steps, the whole group reconvenes to model on the wall, working with potentially many elements.

Subgroups should have no more than three people; this small number allows them to get the work done quickly, encourages everyone to contribute, and discourages one-on-one conversations that can eat up time. Of course, dyads (groups of two) may be even faster, especially at the beginning of a workshop, because they allow the two people to get comfortable with each other. If you have more than six or eight people in your workshop, however, dyads may not be feasible.

Requirements Example: Creating an Actor Map

An efficient way to discover use cases is by using actor maps. An actor map (also called a role map or actor hierarchy) is a diagram depicting how actors relate to each other in terms of shared goals for interacting with the system (see Figure 2). The same actor can appear on multiple actor maps.

Figure 2: Examples of Actor Maps

To generate an actor map using the WoW technique, begin by asking participants to develop a list of actors (a bottom-up wall strategy). Write the actor names on cards, and then arrange them on the wall to form hierarchies. If the same actor appears in multiple hierarchies or maps, then create another card for that actor (see Table 2 for complete instructions).

Table 2: Wall of Wonder Collaboration Pattern: Defining Actors

General Steps

Brief Description

Potential Step Inputs

Focus Questions and Tips

1. Name the actors.

Use the focus questions to list all potential actors. First, have each person create a list on his or her own.

● Stakeholder classes,6 including direct users (actors), customers, suppliers, and tertiary users

● List of project goals and objectives from the charter

● Context diagram7

● Event table8

● Organization chart

● List of use cases

Who directly interacts with the system we're building?Which people and things in our context diagram directly give or get things from the system?What are the subject and object names from our event list?What people or things, such as databases or other systems, have we not yet identified that interact with the system?Given our project goals, who or what

else will need to interface directly with the system? What other systems, databases, or external feeds need to interface with system?

2. List actors on cards in subgroups.

Form groups to collect and document their respective actors, one per card.

● List of actors generated in prior step

3. Place cards from each participant group on the wall.

As a whole group, post cards on the wall one at a time, discussing and agreeing on the name for each actor.

● Cards from each subgroup's work Which card represents the person your group thinks is most critical to achieving our project goals?

Which card represents more of a supporting role with respect to our goals?Which card represents a role you had some debate about?Do you have a card to give me for posting that represents a system or a database?

4. Create hierarchies for shared actor roles.

As a whole group, consider each actor in turn. Use the focus questions to move actors into hierarchies, creating duplicate cards as needed for actors that appear on multiple hierarchies.

● Shared actor roles (generalized actors) from the preceding step

What are the roles we've identified?Which actors belong here? [Cluster the cards together.]Which actors belong there?Do we need to make a duplicate card for this actor?Are there other actors for this role?Where should we put the actors who haven't yet been put into a hierarchy?

5.Summarize and review the hierarchies.

Look for ways to combine, expand, or contract hierarchies to simplify the business.

● All information on the wall

Inspect each of the actor hierarchies.Which actors don't belong?What do the actors in each hierarchy have in common?Do we have all the actors who could play this role?Will we want other actors to also play this role?Are there actors who really should not be playing this role?

6. Finalize the actor map.a) QA the actor map against other models such as the event table, context diagram, or list of use casesb) Reach closure.

a) Ensure that each event has at least one actor who handles the event by initiating a system interaction. Ensure that each actor handles one or more events. Ensure that each actor is depicted on the context diagram. Revise as needed. b) Use the decision rule and decision rule process (see Software Development Magazine, January, 2001) to reach closure.

● Event table

● Context diagram

● Organization chart

● Stakeholder classes

● Use cases

● Organization chart

● Actor map on the wall

● Your decision rules and agreed upon decision rule process

Devise questions that allow you to crosscheck the model. For example, if you have a list of events, ask:Which event(s) does each actor handle?Is there an actor to handle each event?Are there any unnecessary events-ones with no corresponding actor?

Notes for Table 2: Variations and Considerations

● Conduct the entire process as a whole group, particularly if the group consists of seven or fewer people.

● Prepare a draft of an actor map ahead of time to show the participants.

● Identify your subgroups before starting the activity, and decide which groups will get which input model (e.g., events) to work with when generating actors.

● After all the actors are posted on the wall, assign each one to a subgroup; members of this subgroup then form hierarchies independently. Then review the entire set as a whole group.

● After all the actors are posted on the wall, ask half of the participants to rearrange them into hierarchies while the other half takes a break. Then switch so that the people who are back from their break QA the first group's hierarchies, and give the first group their break. Finally, reconvene the whole group to walk through the actor map.

● Avoid using company titles for actor names.

● Suggest naming actors with an "er" at the end, such as "buyer," "stock re-supplier," or "customer query answerer."

● If you have taken a "middle-out" approach and already have a candidate list of use cases (actor goals), then assign a use case to each actor on the actor map, adding use case cards to the map.

I like to test the actor map (or any other deliverable) for "doneness." The doneness test is an agreed-on set of criteria that test whether the deliverable is good enough, and thus whether the group has reached closure. To test an actor map, I often combine a quality checklist with other models --- a list of events, stakeholder classes, or use cases, for example -- to crosscheck the deliverable for correctness, completeness, and consistency. At the end of this process, I ask the group to decide whether the actor map is done by using a collaboration pattern for decision making.9

WoW Warm-Ups

You can jump-start a requirements WoW session with a short imaging activity that helps participants concentrate on what will be important to put on the wall, and a brief mini-tutorial with an example, explaining the requirements' purpose. One essential warm-up is to use focus questions that direct participants' attention to specific items you want to get onto the wall. When the goal is to produce a requirements model, focus questions help direct the group's thinking toward an element of that model. They are also helpful to groups developing an iteration plan, defining a set of project risks, or specifying elements of a project vision.

To model the actors for a context-level use case (context diagram), for example, you could ask, "Who or what provides information to or from the system?" or "Who or what must interact with the system?" When you need to order elements in a model, your question can direct participants to a sequencing task -- for example, "In what order will a user do these steps?"

Focus questions for requirements models can also build on some existing modeling element or project information. For example, if you have a list of actors and want to create a list of use cases, then you can ask, "What goals does the actor have when he interacts with the system?" Or, if you have defined use-case steps and now want to define exceptions, you can ask, "What can go wrong during this step?" This same question is also useful to elicit business rules, because exceptions are business rule violations.

Tools and Equipment for WoW Sessions

I prefer to use low-tech space for a WoW session because it's easy to change, it's dynamic, and frankly it's fun. Keeping your room visually rich and full of tools and materials invites creativity and change.

I equip the space with low-tech materials such as whiteboards, posters, butcher paper, sticky notes or cards, as well as tools such as colored markers. For the surface, I like to use a "sticky wall," which I create by covering two or three walls with poster roll paper that I've sprayed with repositional or remountable spray (available from 3M and other companies -- best to spray these rolls out of doors or where there's plenty of ventilation, then bring them inside to the WoW room). This makes a tacky surface to hold the cards (5X8 colored index cards work well) and lets you reposition them as you like. Alternatively, I use 6x8 sticky notes (repositionable sticky sheets sold by Vis-It or 3M) on a flat wall. An easy way to document the wall's content is to take a digital photo.

As a rule, I don't like to use complex software during the session because it can reinforce tendencies to overanalyze and introduce unnecessary model elements. Projecting word processor content on the wall, on the other hand, can be useful -- especially for text-based deliverables such as use cases, business rules, lists of issues, decisions, and next steps. A person designated as the recorder can enter such text items in real time and project them on a laptop projector, or at least print and distribute them. This is especially efficient during walkthroughs, when the group reviews its work to detect defects.

In one recent workshop, we used both a laptop projector and a printer to great effect. In subgroups, business participants hand-wrote their requirements (in this case, data requirements, business rules, and business justifications) onto a paper template. Then, the documenter entered these into a laptop while the group took a break. The requirements generated by everyone were printed and projected onto the wall for a walkthrough. Instead of waiting to receive corrected documents after the session, the participants made corrections in real time and walked out with a set of requirements.

High-tech tools are useful in a WoW session for both documentation and requirements tracing; the key is to not make the tool the focus of attention. In one workshop, we had an analyst working apart from the group to capture atomic business rules in a requirements management tool. Team members were too busy working with cards on the wall -- building and testing their business rules -- to worry about what the analyst was typing. He printed the rules from the repository in easy-to-read mini-reports several times a day, and the team used these reports, along with previously elicited scenarios, to test each other's business rules in subgroups.

In another workshop, a data analyst observed and recorded the group's data model in his visual modeling tool. This was useful because occasionally the tool triggered awareness that certain elements, such as cardinality and relationship rules, were missing from the wall models. The analyst made a list of these elements, and from this I created a list of quality assurance questions for subgroups to answer.

Nothing beats the speed and throughput of a same time/same place workshop that exploits the WoW approach. But a real-time WoW session also helps set the stage for continued collaboration when the team disperses. That's when automated tools really shine -- by keeping everyone informed about ongoing changes, by linking your requirements to other project elements such as architectural and design models, code, and tests, and by tracing your requirements forward through the development process. In addition, group members, -- who will have established a mutual trust during the workshop, -- can use collaborative software tools for same time/ different place sessions.

Engineering Collaboration into Your Work

Even with all our development tools and technologies, the process of building software still depends largely on the collaborative brainpower of the people who work on the project. The WoW is an effective technique for harnessing that brainpower. It fosters contributions from all individuals and provides a common place for creating emergent deliverables. By employing this simple, easy-to-use, repeatable way to build deliverables, you can accelerate your team's ability to collaborate, and begin engineering collaboration into the software development process.

Acknowledgements

I thank Daryl Kulak, Hong Li, Charlie Poole, Richard Specter, and Karl

Wiegers for their helpful comments and suggestions for this article.

Notes

1 Users, or end users, are the people who will interact directly with the software. Sometimes workshops use surrogates if end users are unavailable, but I make sure they have the content knowledge to truly represent user needs. Indirect users -- people who may not directly use the software - also may have relevant information about requirements (often business rules). Customers pay for the software. It helps if they are present to make decisions about scope and priority as well as trade-offs between functionality and other concerns such as time, cost, or nonfunctional requirements (reliability, speed, and so on). Of course, the people who design and create the software -- including developers, analysts, software engineers, testers, and project managers -- are key participants.

2 B. W. Tuckman, "Developmental Sequence in Small Groups." The Psychological Bulletin, 1976, No. 63, 384-399.

3 See www.icaworld.org.

4 For a complete explanation, see www.ebgconsulting.com.

5 See "Decide How to Decide: A Collaboration Pattern" under Facilitated Workshops at www.ebgconsulting.com.

6 Stakeholder classes (also known as user classes) identify the people who care about -- or should care about -- the systems being developed. Stakeholders include customers, users, and software suppliers, as well as people who have knowledge about the requirements. Stakeholder classes are particularly useful when you're looking to answer questions such as "Who will the system affect?" and "Who will affect the system?".

7 A context diagram (also known as a scope model, system context use case, or context-level use case) shows the system as a whole in its environment. The system is a bubble or oval in the center, external interfaces and actors are boxes and sticks figures, and inflows/outflows are arrows labeled with nouns. A context diagram is useful when you're looking to answer questions such as "What does the system get?" and "When do things happen, and what are the results?" and "What do external actors provide to and receive from the system?".

8 An event table (also known as an event list) defines the "triggers" for events to which the system responses. Some of these responses involve visible outputs; some involve only changes to internal information; some involve both. The table, which lists both the event and the response, is helpful for answering questions such as "When will things happen?"

9 See "Decide How to Decide: A Collaboration Pattern" under Facilitated Workshops at www.ebgconsulting.com.

References

Scott Ambler, "The Practices of Agile Modeling," 2001. See "Display Models Publicly" at http://www.agilemodeling.com/practices.htm.

Alistair Cockburn, "Software Development as a Cooperative Game," 1999. Available at http://members.aol.com/humansandt/papers/asgame/asgame.htm.

Ellen Gottesdiener, Requirements by Collaboration. Addison-Wesley, forthcoming in 2002.

Institute of Cultural Affairs, 1994. Group Facilitation Methods course guide, Institute of Cultural Affairs.

Jon R. Katzenbach and Douglas K. Smith, The Discipline of Teams: A Mindbook-Workbook for Delivering Small Group Performance. John Wiley and Sons, 2001.

Michael D. Schrage, Serious Play: How the World's Best Companies Simulate to Innovate. Harvard Business School Press, 1999.

Laura Spencer, Winning Through Participation: Meeting the Challenge of Corporate Change with the Technology of Participation. Kendall/Hunt Publishing Co., 1989.

Richard A. Zahniser, "Design by Walking Around." Communications of the ACM, October 1993, pp. 115-123.

For more information on collaborative workshops to create requirements deliverables, also see The Rational Unified Process, "Work Guidelines for a Requirements Workshop" and "Work Guidelines for a Use Case Workshop."

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Progressive Acquisition and the RUP: Comparing and Combining Iterative Processes for Acquisition and Software Development

by Gilles Pitette

Software Engineering SpecialistRational Software, France

I could tell you many sad stories about efforts to acquire large and complex information systems using classical, strictly sequential acquisition models that ultimately ended in failure. In recent years, this poor record has led many organizations -- especially in the military -- to adopt alternative, iterative approaches to acquisition. This article looks at one such approach in detail and explores how it relates to the Rational Unified Process® (RUP®).

Several years ago, a group of European nations formed a committee to investigate ways to improve the acquisition process for Defense Information Systems (DISs). The

result was a publicly available document on Progressive Acquisition (PA).1 In this article, I use PA as an example of an iterative approach to acquisition and as a point of comparison with the RUP, often borrowing material from the committee document.

Acquisition Versus Development

The PA document defines the acquisition process as:

... the process for buying a system, software product or software service and, generally, all activities undertaken by the purchaser during the entire lifecycle associated with the existence of an information system, from the initial step to the final one. For example: budget, study, tender and contract, development, quality control...

In the context of PA, an acquisition has three major stakeholders:

● The Acquirer: the "organization that acquires or procures a system, software product or software service from a supplier."

● The Supplier: the "organization(s) responsible for delivering a system according to its requirements."

● The User: a "person who will make direct usage of a system (User), or will be responsible for people making such direct usage (Support User)."

In some contexts, the term procurement may be used to denote a concept similar to acquisition. For PA, procurement is limited to "the range of activities associated with contracting the delivery of an information system" and is simply "a part of the acquisition sequence."

Overall, this definition of acquisition assumes that some form of contractual relationship is established between the acquirer and the supplier. Moreover, software development is understood as "a sub-process of the overall acquisition process."

Contracts may be established at many different points in the acquisition of an information system, and the object and nature of these contracts may vary widely. For example, the acquirer may contract with an individual expert for a certain number of days of consulting to address specific issues during the early phases of an acquisition. Or, once system requirements have been specified, the acquirer may select a prime contractor to conduct all further phases of acquisition up through system delivery.

For the sake of simplicity, let's assume that the acquisition and development processes correspond to the two sides of a customer/supplier relationship:

● The acquisition process describes all activities of an acquisition team in procuring a system for the benefit of its users. This includes activities relating to contracting during the development process (e.g., requirements definition, contract establishment, and contract execution).

● The development process describes all activities of a development team for constructing and delivering the system to the acquirer.

Improving the Acquisition Process

Over the last twenty-five years, the software industry has devoted much effort to improving the way software developers construct information systems for their customers. In addition to considerable advances in domains such as programming languages and methods, improvements in the software engineering discipline have yielded widely accepted best practices and supporting tools.

On the acquisition side, however, the advances have not been as striking. To meet today's twin demands for higher quality and faster time-to-

market, the development and acquisition processes should be aligned, and we now recognize the need to improve both acquisition practices and development practices. Recent initiatives include the following:

● In the US, software development maturity models (e.g., the Capability Maturity Model,®2 or CMM) have been elaborated to cover the acquisition process; some actually integrate acquisition and development into a single model.

● In Europe, SPRITE-S23 is an initiative of the European Commission that focuses on improvement in procuring IT systems and services. Some of the SPRITE-S2 projects have collected acquisition best practices, proposed an extension4 to the ISO/IEC 15504 standard (initially developed by the SPICE5 project) in order to cover the acquisition process, and developed services for benchmarking this process. Further, the Procurement Forum6 has been established as a place where procuring organizations can share their experiences.

● The Western European Armament Group (WEAG7) TA13 Acquisition Programme is a co-operative effort of Western European ministries of defense. Its aim is to improve the acquisition of DISs via the adoption of novel iterative approaches, and it produced the document that defines and guides the application of PA.

The Progressive Acquisition Approach

As I said at the beginning of this article, there are many sad stories to tell about large DIS projects procured under the classic, strictly sequential, "big-bang" model (also termed "grand-design" or "one-shot") -- stories about late deliveries, cost overruns, and failures to meet users' real needs. The problems stem from the big bang model's inherent inability to deal with a few stark realities:

● You can't express all your needs up front. It is usually not feasible to define in detail (that is, before starting full-scale development) the operational capabilities and functional characteristics of the entire system. These usually evolve over time as development progresses.

● Technology changes over time. The DIS acquisition lifecycle spans a long period of time during which, given the pace at which technology evolves, significant technological shifts may occur.

● DISs are very large and complex systems. This means it is difficult to cope with them adequately unless you have an approach for mastering complexity.

In the late Nineties, the organization devoted to Software Engineering within the Western European Armament Group (WEAG TA13) launched a program for improving DIS acquisition by providing guidance on the use of IA/EA (Incremental Acquisition/Evolutionary Acquisition). Participants in this co-operative effort included France, Germany, Italy, and the United Kingdom, with Sweden as an observer.

Along with new studies of IA/EA approaches in participating nations, in the US, in NATO, and even more widely in the literature, came much confusion about what distinguishes an incremental from an evolutionary approach. In practice, a combination of the two is very often desirable, and PA provides such a "unified" approach.

The PA document introduces the approach in the following way:

PA is a strategy to acquire a large and complex system, which is expected to change over its lifecycle. The final system is obtained by upgrades of system capability through a series of operational increments. PA aims to minimize many of the risks associated with the length and size of the development, as well as requirements volatility and evolution of technology.

An essential goal of PA is the rapid fielding of a usable system that not only addresses an initial and validated statement of needs, but also anticipates iterative upgrades of system capability as development progresses through a series of increments. Each successive increment yields an operational version of the system that meets a pre-specified subset of the overall system requirements.

Continuous user feedback based on previously fielded increments is an essential element of the approach. It may significantly influence the definition and development of later increments. In the same way, technology updates may be accommodated across increments, reflecting the evolution or obsolescence of hardware and software items, including commercial off-the-shelf products.

Key elements of the PA approach include the following:

● An Overall System Concept describing the desired final system capability (including operation and system functional requirements in broad terms). This is defined by user representatives.

● A System Architecture capable of accommodating system evolution with minimum redesign.

● A PA Plan defining the progressive achievement of final system capability through iterative development. This includes the fielding and support of successive increments, each providing upgrades to the system operational capability.

● A detailed operational concept for a Core Increment of the system (developed in parallel with the above elements):

■ Delineates the subset of the Overall System Concept to be implemented in the Core Increment.

■ Refines requirements for the Core Increment as appropriate.

● The Core Increment is then developed, tested, fielded, and sustained by an in-service support unit.

● The Core Increment is then operated by the user in the operational environment; the user provides feedback to be considered in the

definition of later increments.

● The following successive increments are then iteratively defined, approved, developed, tested, fielded, and supported in the same way as the Core Increment. Variations may occur in the way successive increments are distributed over time, depending upon the specific needs of the project. Successive increments may overlap, be strictly consecutive, or possibly even be staggered in time.

● This approach continues until the final system configuration is achieved.

Figure 1 shows an overview of the PA approach, including its iterative application over time.

Click to enlarge

Figure 1: Overview of the Progressive Approach8

The Progressive Approach's Two Dimensions

PA proposes a common progressive philosophy that is flexible in nature and must be tailored to the needs of each specific acquisition. It defines a framework within which acquisition is characterized by two complementary dimensions: incremental and evolutionary.

The Incremental Dimension. The incremental dimension of PA is a means to reduce the inherent risks associated with the length and size of the development of large and complex DISs. The incremental qualifier refers to the staged planning, development, and delivery of system capability, considering an essentially stable set of requirements. It means the system is partitioned into successively developed, tested, and fielded "increments," with each increment building upon the previous one until the entire system is complete (see Figure 2).

Figure 2: Incremental System Growth9

The Evolutionary Dimension. The evolutionary dimension of PA is essentially a means to reduce the risks associated with the expression of DIS needs. Often it is not feasible to define the full requirements of a project in detail at the outset. In addition, certain requirements may be highly unstable over time. The evolutionary qualifier refers to requirements for system capability, which will evolve over time. The degree to which an acquisition is evolutionary depends on the project's openness to user feedback as well as on the degree of significant technology evolution across successively developed, tested, and fielded system increments (see Figure 3).

Figure 3: Evolutionary System Growth10

Combining Incremental and Evolutionary Dimensions. Many acquisition projects combine the incremental and evolutionary dimensions. The relevance of each depends on the project's specific risks or characteristics, and PA should be "tailored" to meet the project's specific needs along these dimensions.

This need for tailoring also applies to defining the successive increments established in the PA Plan, which must state:

● Which elements of capability will be delivered by each of the successive increments (this also defines the extent to which the acquisition is incremental);

● How evolution of previously delivered capability is accommodated across remaining increments (this also defines the extent to which the acquisition is evolutionary).

This tailoring of PA should be driven by an analysis of the subject project. Relevant characteristics of the project need to be assessed in order to identify risks threatening the acquisition as well as opportunities for PA to address those risks. The uncertainty and instability that characterize many DISs affect project elements. Also, feedback from users and the technology during the acquisition may call for modifying elements that influenced initial project planning, possibly including the system architecture. This means that PA re-tailoring and updates to the PA plan may be necessary during the course of a project and should be considered before initiating a new increment.

Process

The PA Process defines the sequence of activities needed to implement the PA approach as well as the contents of the work products associated with these activities.

Figure 4 illustrates the various activities and work products in a PA sequence. Note that only those that are specific, or essential, to PA are shown (e.g., the DIS itself is not shown as an output work product).

Figure 4: The PA Sequence of Activities10

Compared to other acquisition approaches, PA is essentially an iterative approach involving a series of increments that are successively acquired. The activities internal to the acquisition of a single increment are similar to those in a "big-bang" acquisition; in describing the PA sequence, they can be considered as a single activity.

Activities in the PA sequence are three-fold:

● Upstream activities, at the beginning of the acquisition, which result in the Overall System Concept -- an initial input for the rest of the acquisition sequence.

● Long-term activities, before entering the acquisition of the core increment and possibly at each iteration, which result in the System Architecture and the PA Plan that will drive the rest of the complete DIS progressive acquisition.

● Iterative activities, for each increment, which allow the acquisition of a single increment. Also, when evolution is afforded by the PA Plan, these activities allow operation of the delivered system, collection of Feedback from users' experiences, and evolution in requirements and technology, which may influence future increments.

Upstream Activities. The Overall System Concept describes the user's needs, goals, expectations, operational environment, processes, and characteristics for the planned DIS. It should be written in the language and terminology of the user's application domain. The subset of the Overall System Concept to be implemented in each increment is refined into requirements prior to the development of that increment. The Overall System Concept should contain items such as:

● An overview of the capability desired for the full system in functional terms.

● A description of the current system or situation.

● A description of the needs that motivate the acquisition.

● Operational scenarios for each operational mode and class of user.

● Operational features and their priorities for the proposed system.

● Reliability, security, and availability objectives for the proposed system.

● Connectivity and interoperability to other projects, external systems, and interfaces and system boundaries in capability terms.

● Limitations and risks of the proposed approach.

● Impact analysis for the proposed system.

Long-Term PA Activities. The System Architecture is a critical element that should be carefully defined, although a high degree of specificity may not be possible initially. It must be designed to be capable of accommodating system evolution with minimum system redesign. Among other aspects, it should describe:

● The functional and technical architecture.

● The relationships between architecture elements and requirements (traceability).

● The use of off-the-shelf products.

The PA Plan is a high-level management plan for a project under PA; it covers the entire lifecycle and provides a long-term perspective with sufficient flexibility to accommodate adjustments dictated by user feedback and technology evolution. The PA Plan must be revised iteratively as a result of project progress. It should describe, at a minimum:

● The definition of a series of increments: capability, schedule, and budget for each increment.

● The nature of user feedback accommodated between increments: corrective actions versus capability modifications or enhancements.

● The treatment of technology evolution: openness to integration of new technologies.

Key project organization elements: stakeholders and their relationships, roles, contracting policy, reuse, standards, and tools policy.

Recommendations

The PA document provides an extensive set of recommendations, grouped by themes, on how to put PA into practice. These recommendations are intended for project managers responsible for acquiring a DIS. They address issues that either are specific to PA, as compared to more traditional acquisition models, or that have a higher relevance under PA.

Contracting is one of the themes addressed by these recommendations, as administering contracts in conformance with the PA approach may be very

difficult under well-established contractual models.

Comparing PA and the RUP

The customer-supplier relationship established between the acquirer and the developer of a DIS is integral to the acquisition process. Progressive Acquisition, along with the guidance for its use, is designed for Project Managers on the customer side of the acquisition process.11

The Rational Unified Process, in contrast, is for those on the supplier side. The RUP defines itself as "a software engineering process" that provides a "disciplined approach to assigning tasks and responsibilities within a development organization," while aiming to "ensure the production of high-quality software that meets the needs of its end users within a predictable schedule and budget."

The RUP integrates modern software development best practices and defines an approach to software development that is iterative, architecture-centric, and based on use cases to manage requirements. It does not distinguish acquirers from suppliers as different stakeholders in the process, and it does not distribute responsibility for the activities in the process across these two roles. As a consequence, contracting issues are largely outside the scope of the RUP.

Let's look at some specific points of comparison between these two processes.

Phases, Iterations, and Milestones

Iterations, as opposed to a pure (one-shot or single iteration) waterfall approach, are central to both the PA and the RUP. However, both processes also recognize the need for sequential steps at the coarse-grain level of project execution. In the RUP, these steps are formalized as the Inception, Elaboration, Construction, and Transition phases.

The PA process, as we saw in Figure 4, informally structures its activities into sequential groupings of upstream, long-term, and iterative activities.

● Upstream activities result in the Overall System Concept, which defines expectations for the system to be acquired.

● Long-term activities produce the System Architecture and PA Plan, both setting the basis for later iterative activities.

● Iterative activities (performed during each iteration) include the acquisition of a system increment, its operation by users, and the collection of feedback from its usage.

The scope of PA upstream activities is similar to that of the RUP Inception phase. Their output (Overall System Concept) addresses some of the concerns of the RUP lifecycle objectives milestone (that is, a global vision of what the system should be). However, the PA document, unlike the RUP, does not formally specify that upstream activities should be done in

several iterations.

The scope of PA long-term activities is similar to the RUP Elaboration phase. Their output (System Architecture and PA Plan) addresses to some extent the concerns of the RUP lifecycle architecture milestone (that is, the architecture of the system and a plan for future activities). The PA does not specify that long-term activities be done iteratively. It should be noted, however, that the recommendations for using the PA do suggest experimental prototypes for designing the architecture. Furthermore, these recommendations also advise demonstration and delivery of the technical architecture as early as possible.

For each increment, we can think of the scope of PA iterative activities as roughly equivalent to an aggregation of the RUP Construction and Transition phases, but without internal iterations. In contrast to the RUP internal iterations, each PA system increment is considered to be an operational system (with the quality required for operation): It is delivered, fielded, and operated by end users in an operational environment.

Disciplines

The PA document includes an acquisition framework, which lists activities usually performed during a DIS acquisition. These activities represent themes of work for the acquisition team, independent of when this work happens in a PA sequence.

The top-level items in this framework are:

● Acquisition preparation

● System requirements management

● Contracting

● Project management

● Acceptance

● System operation

Such items are similar to the RUP disciplines, but specialized for the acquisition side12. The big difference between acquisition framework items and the RUP disciplines is that the PA document only identifies and defines framework items to be used as entry points for navigating the set of recommendations for the use of PA provided by the PA document; the RUP, in contrast, provides a detailed specification of the disciplines, and of the activities they contain.

Architecture

Both approaches highlight the criticality of the software architecture to the success of the project.

Further, the PA recommendations suggest that the architecture should be

"mastered and controlled" by the acquirer -- that is, the acquisition team should understand it, have a long-term view of it, avoid solutions proprietary to particular suppliers, and so on.

Risk Management

With the RUP, risks play a major role in planning project iterations. Highest risk items should be identified and addressed early in the project -- i.e., early iterations should aim to mitigate and reduce them.

Overall, the PA approach specifically aims to minimize two types of risks:

● Those associated with the duration and size of the development project for a large and complex system; these should be reduced by adopting the incremental dimension in the acquisition.

● Those associated with requirements volatility and evolution of technology; these should be reduced by adopting the evolutionary dimension in the acquisition.

Under PA, addressing high risks up front is not necessarily what drives the planning for acquisition increments. In contrast to the RUP, the PA evolutionary dimension encourages acquirers to quickly field a system that addresses whatever requirements are best known at the outset of the project. Nevertheless, if you do apply the PA incremental dimension, those risks specifically associated with the system architecture should be addressed in early increments.

Mapping PA and the RUP

The fact that both PA and the RUP advocate an iterative approach raises a set of questions: Do these two models fit together? How could they collaborate in a software project?

At first glance, the most "natural" way to map PA and the RUP would be to think of PA increments as RUP generations. Within this framework:

a. PA would be used by the acquirer to iteratively obtain a series of increments (generations) of the desired software system.

b. The RUP would be applied by a supplier to develop a single generation (increment) of that system; that is, the Core Increment would be the first generation/iteration, then each successive increment would correspond to phases and iterations in the RUP evolution cycle.

Such an approach would include two nested levels of iteration in the execution of a software project: a series of increments at the acquisition level managed with PA, and a series of iterations at the development level managed with the RUP.

However, in such a scheme, these two levels would be loosely coupled; each would remain "independent" from the other (see Figure 5). In

considering a single increment, the classical relationship between acquisition and development activities would remain intact, even though the RUP iterative process would be used for developing the software corresponding to this increment. Further, any other development process (even a traditional waterfall one) could perfectly well be used for the development of an increment, with no impact on other increments. Conversely, developing an increment iteratively with the RUP would not necessarily have an impact on the development of other increments.

Click to enlarge

Figure 5: Nesting the RUP in PA

Reality Is More Complex

Such a straightforward mapping of PA and the RUP leads to the implication that the RUP is useful only for pure software development organizations (that is, organizations that contract to develop software systems), and not for organizations acquiring software systems. In practice, however, certain disciplines (activities) and artifacts that appear in the RUP are actually more within the scope of the acquisition team, or at least should be performed under its responsibility:

● Business modeling, as proposed by the RUP, is something that should take place during PA upstream activities, when the Overall System Concept is defined. Such modeling must involve various stakeholders in a co-operative effort: users of the system, the acquisition team, and possibly a supporting contractor (possibly the same as the supplier -- or one of the suppliers -- who will be selected for delivering the upcoming increment[s]).

● Architecture is equally critical for PA and the RUP. The System Architecture (as seen under PA) must be elaborated up to a certain level of detail during PA upstream activities. Then, it should be revisited during the Elaboration phase of each subsequent system generation/increment only if necessary. The definition of System Architecture may be contracted to a supplier, perhaps the same supplier(s) that will be selected for delivering the upcoming increment(s). However, it must also be mastered and controlled by the acquisition team.

Some other disciplines (activities) or artifacts should be performed or produced in both processes, even if their scope may vary between acquisition and development:

● Requirements Management is of paramount importance under PA. The acquisition team has to define initial requirements, at least to a certain level of detail, and feed them to the development team. Then it has to manage requirements evolution across increments. However, within the development of a system generation, requirements may need to be refined, and thus managed by the developer. Overall, artifacts defining requirements obviously are to be worked on by both the acquisition and development sides.

● Change and Configuration Management is necessarily related to requirements management. The acquisition team has to manage change of requirements across PA increments. It also has to manage the traceability across changing requirements and successive generations (increments) of the system. At a more detailed level, the iterative nature of software development with the RUP requires a very disciplined and tooled approach to change and configuration management.

● Test Management concerns both the acquisition and development sides. In the acquisition process, the acquisition team has to manage, at a minimum, acceptance tests (possibly contracting them to a different team than the development team). The development side has to manage other types of tests (for example, unit testing).

These elements show that simply nesting the RUP within PA is not sufficient. With iterative approaches, reality is more complex, and acquisition and development activities are more closely coupled than they are with a waterfall approach.

The RUP for Acquisition Organizations?

The PA document explicitly states: "Combined user/customer-supplier effort is a key ingredient of PA." The level of coupling between acquisition and development that we can see when we nest the RUP in PA indicates that it is highly desirable to apply common practices on both sides:

● For modeling the user's business.

● For modeling the architecture.

● For managing requirements.

● For managing change and configuration.

● For managing tests, and so on.

Unifying the practices of the acquirer and the supplier(s), and viewing the two sides as a team, enables more efficient communication and co-operation. Unifying the tools that are used to support these practices is also a must.

So that leads to another question: Should the PA and the RUP be integrated? Clearly, those on the acquisition side would benefit from using the RUP along with an acquisition approach. The new acquisition

approaches we've been discussing are very general frameworks that introduce an iterative philosophy, in contrast to classical acquisition models that use a mostly sequential, waterfall approach. However, they do not go far enough. The PA document does provide general concepts and guidance to acquisition project managers on how to implement an iterative (progressive) approach, but it does not describe the PA process with the level of detail appropriate for direct use by acquisition project managers.

This is in sharp contrast to the amount of material and guidance the RUP provides for software developers on the RUP Web site13. Plus, although the RUP does not separate the respective concerns of acquirers and suppliers, it does fully integrate all the ideas that motivated people to create the new, iterative approaches to acquisition. The most important messages conveyed by the RUP (for example, the imperatives to work iteratively and to be architecture-centric) may be understood by both communities.

The RUP, or parts of it, could therefore either be profitably combined with an approach like PA or used alone by acquisition teams needing a detailed, tool-supported process for managing iterative software projects:

● To model business processes with the Unified Modeling Language (UML).

● To express requirements with use cases, and manage them appropriately.

● To model the system architecture with the UML.

● To manage change and configuration.

● To manage test.

Using the RUP for the acquisition side would also require two changes:

1. Separating acquirer and supplier concerns in the RUP disciplines.

2. Introducing issues related to contracting and the acquirer/supplier relationship in the RUP.

Contracting may occur at many different junctures in the acquisition process. Moreover, the scope and nature of contracts may vary considerably. Introducing contracting in the RUP could make it more complex, so any attempt to do so should include measures to keep the RUP "usable" for any other target user.

The RUP: A Way to Unify Acquisition and Development

The emergence of the new approaches we've discussed shows that acquisition organizations have recognized the need for an iterative process, and that process should include best practices and tools in areas such as business modeling, architecture, requirements management, configuration and change management, test management, and so on.

It would be a serious mistake for acquisition organizations to think of the RUP as a tool for development organizations only. The RUP can be extremely useful to those organizations, even if they contract their development to external suppliers.

Further, unifying the practices and tools of acquisition organizations with those of their software development suppliers can be key to the success of a project. Understanding this, acquisition organizations may actually prescribe practices and tools for their suppliers.

Notes

1 WEAG TA-13 Acquisition Programme. Guidance on the Use of Progressive Acquisition, Version 2. November 2000.

2 See http://www.sei.cmu.edu/cmm/cmm.html.

3 Support and Guidance to the Procurement of Information and Telecommunication Systems and Services. See http://europa.eu.int/ISPO/topics/i_standardisation.html.

4 See http://wwwsel.iit.nrc.ca/spice/Viewer.mv?TARGET=AMEND

5 Software Process Improvement and Capability Determination project. See http://www.sqi.gu.edu.au/spice/

6 See http://www.procurementforum.org/

7 See http://www.weu.int/weag/

8 From WEAG TA-13 Acquisition Programme, Op.Cit.

9 Ibid.

10 Ibid.

11 Customers should note that PA's iterative nature actually enables the acquirer to change suppliers between two successive increments.

12 Unfortunately, the PA document does not depict them as neatly as the RUP depicts its disciplines (see the introductory figure on the RUP home page).

13 Instantiation of the general approach of PA into a well-defined process has been left by the WEAG TA13 Acquisition Programme to a national level. Such instantiation must take into account the corpus of standards, rules or guidelines existing locally.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Project Scope Management: Effectively Negotiating Change

by Ben LiebermanSoftware Architect

All projects begin with the definition of needs. Needs of the client, needs of the development organization, needs of project managers, programmers, architects, and more. These needs cannot all be met in a reasonable time frame. In order to make progress, these needs must be prioritized in some way. Often, the project manager is responsible for establishing the base project priorities.

A good first step is for the project manager to prioritize all the identified needs, which we refer to as establishing the "project scope." If the project scope is too large, then the development organization will be overwhelmed, leading to chaos and project failure. If the scope is too narrow, then the resulting system will be unfit for the desired purpose, and the customer will likely refuse payment. Effectively balancing these varied needs requires all parties to negotiate in good faith and adopt a cooperative model between the client and the software developer. This article presents techniques for effectively categorizing, organizing, and prioritizing requirement changes over the course of a project; it also suggests ways to use this information to conduct effective negotiation between the vendor and the client.

Software programs are conceived to solve specific problems. An accounting package may provide the ability for businesses to perform complex accounting practices. A scientific software package may allow a researcher to conduct a sophisticated analysis of experimental data. A sufficiently full-featured software system can fly a spacecraft. In order to create a software application, a development organization needs to define the problem, interview users to discover their specific needs, and establish a project scope for development. While determining the system scope, the project manager must walk a fine line between the client, who desires the system to be as useful as possible, and the developer, who must create

the system within a fixed timeframe and budget.

All truly successful projects are collaborative efforts between the customer and the development organization. Collaboration allows for increased efficiency, accuracy, and comfort for all participants. Unfortunately, most projects operate using a competitive model that creates an unhealthy tension between the client and the development organization; this leads to disagreement, argument, even lawsuits. Along the way, the original goal of creating a software system that meets the true needs of the user is lost.

Below, I will first examine a couple of key factors in the client/vendor relationship, taking into account human nature and the effect the desire to "have one's way" can have on the relationship. Then I will present some techniques for analyzing the impact of scope changes on the project schedule and prioritizing requirements based on cost, risk, benefit, and effort.

Cost of Features

All software features cost a significant amount of money to develop, and controlling cost is one of the jobs of project management. Often, development teams respond to pressure to please a critical client (who is, after all, the source of the money) by allowing the introduction of new requirements into the project. Unless project scope can be effectively managed, such additional requirements may flow into the project in a haphazard manner, leading to the inclusion of features that were neither discussed nor planned. It is likely that these features will not be properly tested (if they even get mentioned to the test team), and the addition of this functionality will likely come at the expense of the overall system architecture. Of course, some changes will be necessary and beneficial, but many will be detrimental.1 In addition, the haphazard introduction of new features will destroy a project schedule more rapidly than any other single cause.

As illustrated in Figure 1, satisfying 1% additional needs can easily translate into 10-50% additional development work. This dramatic increase is due to the work needed to correctly capture and communicate the desired features, translate these features into a workable computer architecture, and finally create complex programming code.

Figure 1: Requirements Cost Pyramid

On the other hand, good scope management can significantly reduce the workload by eliminating or postponing a very small percentage of "needs." A project manager can avoid success-threatening work by reaching an agreement with the customer on the true importance of a requested change. Naturally, clients seek to maximize delivered functionality for minimum cost, just as development organizations seek to minimize risk and maximize return on investment. What's needed to resolve this dilemma is the ability to negotiate. To negotiate effectively, we need to understand the likely impact of each feature in terms of risk, effort, and benefit.

Effective Negotiation

The goal of negotiation is to resolve competing forces, bringing parties into agreement on a reasonable solution to the conflict. Most often a conflict arises because of the need for each party to look for the most advantageous personal position, the standard driving force of competition. Although competition between groups can lead to positive gains between two organizations, competition within a group is almost always detrimental. A software project requires a close, cooperative alignment between the client and the vendor, so an understanding of the forces that will affect this alignment is beneficial.

Understanding the Game

Many of the forces driving individual or group actions during conflict can be modeled through "Game Theory," which uses mathematics to help competing parties choose an optimum strategy.2 A classic example of the conflict inherent between cooperation and competition is evidenced in the simple but profound game called "The Prisoner's Dilemma," invented at the Princeton Institute of Advanced Science in the 1950s.3

The game begins when you and your accomplice are captured following the commission of a crime. The police separate you from your partner and offer you the chance to "give evidence" against your partner. You are told

that she will be given the exact same deal in a separate room. Figure 2 shows the matrix describing the result of your actions and those of your partner (the numbers represent years in prison):

Figure 2: The Prisoner's Dilemma Matrix

If you "stay silent," then you are attempting to cooperate with your partner to avoid a long prison term. If you refuse to give evidence against your partner, and she does the same, then you both receive one year in prison (Cooperative Payoff). If, however, you keep quiet and she gives evidence, or competes with you, then you go to jail for five years and she is set free (the so-called Sucker's Payoff). Finally, if you both "give evidence" against one another, then the judge sends both of you to jail for three years each (Competitive Payoff). The dilemma resides in the fact that each prisoner has a choice between only two options, but cannot make a good decision without knowing what the other one will do. So what choice should you make? When should you cooperate, and when should you compete?

Conflicts of this nature often occur when two groups are attempting to work together. In software development the two groups are the vendor and the client. The relationship often starts out with good feelings and high expectations from both parties. However, as the project progresses and problems are encountered, communications between the client and the vendor tend to deteriorate. This leads to the situation outlined above: both sides have the option of continuing to cooperate or compete, but neither can make a decision without being able to trust the other party.

Software development is often a case of multiple rounds of The Prisoner's Dilemma. To map this game into "The Software Developer's Dilemma" (Figure 3) we can reassign each sector in the payoff matrix:

● The development organization often wishes to solve the problem in a particular way, regardless of what the customer thinks (Dictate).

● Alternatively, the customer may choose to force a pet feature down development's throat or threaten to pull out of the project (Extort).

● If both parties argue, then very little progress is made (Impasse).

● Only through cooperation is a significant benefit for both parties guaranteed (Support).

Figure 3: The Software Developer's Dilemma Matrix

What occurs as the level of trust in the relationship breaks down, most often as a result of a crisis, is that one side or the other begins to seek greater personal advantage. Then, to avoid the "Sucker's Payoff," the other side will move to a competitive stance. This tit-for-tat response leads inevitably to Impasse, until one side gains sufficient power to force its position (i.e., Dictate, Extort). The game is played repeatedly during the life of the project until very little trust exists and all decisions require a large expenditure of time, effort, and emotional capital.

Congruent Negotiation

It should be clear from this model that cooperation between parties is the best choice. So how do we avoid the feedback loop that leads to competition and sub-optimal returns? The key here is good and regular communication between the vendor and the client. Remember, the "dilemma" in both The Prisoner's Dilemma and The Software Developer's Dilemma is caused by the inability of one party to know what the other party is choosing to do. To ensure that mutual benefit remains the choice, it is necessary for both parties to congruently (openly and honestly) negotiate all desired changes to the system.4

As opposed to competition, congruent negotiation is seeking a win-win for both parties. No side should seek unfair advantage over the other, because that will lead to sub-optimal long-term gains. In fact, both parties should be clear on the cost of non-cooperation and the negative impact competition will have on the probability of project success.

Here are some guidelines on conducting effective congruent negotiations:

● Establish mutual benefit as a motivational force.

● Negotiate from a position of openness.

● Stay congruent, express disagreement honestly.

● Avoid pointless argument.

● Be willing to compromise, but don't placate.

● Seek a neutral mediator when faced with an impasse.

Business relationships being what they are, during negotiations it is very good practice to establish protective measures for each party to ensure compliance to the contracted agreement. This can be thought of as ensuring the relationship and maintaining cooperative behavior for the best possible outcome. In his poem "Mending Wall," Robert Frost expresses this another way: "Good fences make good neighbors."5

Here are some suggested practices that can be used to protect both the vendor and the customer. This list is only a sampling of the possible mechanisms:

To protect the developer:6

● State in the contract that the development house owns the code until final payment.

● Agree on clear and reasonable acceptance criteria (given that no software is "defect-free").

● Include a software key that will operate after the date of contracted software acceptance. (This must be clearly disclosed in advance to avoid legal issues.)

To protect the client:

● Clearly state that payment will be provided only for software that meets the agreed upon functionality (e.g., based on use cases).

● Require milestone presentations of progress for continued funding (i.e., functioning executable releases, not just documentation).

● Provide a realistic expectation of software reliability and functionality.

Scope Impact Analysis

As noted earlier, changes to the project scope can and will have a large impact on the success of the project. There will always be the need to accommodate change, but without analyzing the cost and benefits of the desired change, rational decisions cannot be reached. Effective negotiations require not only an understanding between the parties, but also accurate, focused information about the impact the changes will have on the project scope.

Scope is established early in the Inception phase of the project. In order to effectively negotiate the scope, it is necessary to have accurate and useful information on the relative costs and benefits of a requirement. During the project it is of even greater importance to determine the effect of a requirement change on the system under development. To determine the likely impact of additional work, it is necessary to understand the risks, benefits, costs, and effort associated with the change(s). When dealing with requirements it is beneficial to:

● Categorize -- to group requirements, permitting a higher-level understanding of relationships and dependencies; use cases are the preferred mechanism.

● Organize -- using automated tools (e.g., Rational RequisitePro®) to assist in understanding and tracking of the requirements from introduction to acceptance to implementation.

● Prioritize -- to determine the order of consideration based on criticality of need and level of associated risk.

According to the Rational Unified Process®, a project consists of four primary phases, Inception, Elaboration, Construction, and Transition. Inception defines the project goal and initial scope; Elaboration is primarily concerned with system architecture; Construction is focused on creating the final product; Transition is moving the product into the hands of the customer. As can be seen in Figure 4, the bulk of the requirements will be gathered in the Inception and Elaboration phases. However, it is during Construction when changing or additional requirements are most likely to have a large system impact and must be carefully studied for risk, particularly to the system architecture.

Figure 4: Project Phases and Requirements Load

Requirements are categorized according to Risk, Effort, Priority, and Cost.

Effort is the amount of manpower that must be expended to accomplish a task and can be estimated from a well-developed set of use-case activity diagrams (i.e., by totaling activities),7 or by counting the number of pages in a use case (not including title, header, etc.). Alternatively, a fast but qualitative estimate can be made directly by experienced personnel.

A risk -- any situation that can negatively impact the success of the

project -- is affected by numerous factors, including:

● Political sensitivity

● Technical difficulty

● Effect on team morale

● Client perception of progress

● Management commitment

● Impact on schedule

● Impact on cost of delivery

● Impact on quality of product

● Effect on system architecture

Of these, perhaps the most damaging is the effect on system architecture. As with smoking, the first short cut is unlikely to kill you, but multiple architectural compromises over the life of the project will inevitably cause serious damage.

Cost is determined by the amount of money/time that must be spent to produce the required functionality. This is therefore a combined factor of effort and schedule.

Priority is usually determined by the customer, but may be determined by the project development team for internally generated system changes.

Use cases (and requirement changes) can be rapidly rated on these categories using a simple scale (High, Medium, Low), as shown in Table 1.

Table 1: Use Case Categorization and Prioritization8

Use Case Effort Risk Cost Customer Priority

UC1 H H H 1

UC2 H L L 6

UC3 L M M 5

UC4 M H L 3

UC5 M L H 4

UC6 L H L 2

The determination of risk is based on the overall success of the project, as opposed to just one system feature. When determining the relative ranking, it is therefore useful to prioritize based on highest risk first, followed by effort and customer priority.9

Using this technique allows the project manager to negotiate with the customer based on a solid, qualitative analysis of the relative benefit and risk for a given requirement change. Attacking the high-risk elements early leaves more time in the schedule to deal with contingencies. Moreover, items that have high risk with little direct benefit can be eliminated from the project or delayed to later releases of the software.

Organizing use cases in this fashion will facilitate negotiations by allowing

tradeoffs among risk, effort, and cost.

Effective Scope Management

All software projects must deal with changing requirements. The most powerful tools available to the software organization are effective negotiation skills backed by organized and prioritized requirement sets. Establishing a cooperative environment with agreed-upon verification mechanisms protects both the client and development organization, thereby increasing the level of trust. The result is a project scope that will meet the most critical needs of the client, while permitting the development team sufficient flexibility to produce the software in a timely manner.

Notes

1 T. F. Crum, The Magic of Conflict. Simon & Schuster, 1987.

2 "Game theory is a branch of mathematical analysis developed to study decision making in conflict situations. Such a situation exists when two or more decision makers who have different objectives act on the same system or share the same resources. There are two person and multiperson games. Game theory provides a mathematical process for selecting an OPTIMUM STRATEGY (that is, an optimum decision or a sequence of decisions) in the face of an opponent who has a strategy of his own." from PRINCIPIA CYBERNETICA WEB, http://pespmc1.vub.ac.be/ASC/GAME_THEOR.html

3 Robert M. Axelrod, The Evolution of Cooperation. Basic Books, 1984.

4 Congruent communication involves expressing yourself honestly and directly, as opposed to non-congruent behavior, which includes blaming, placating, irrelevancy, etc. For more information on congruent communication, see G. M. Weinberg, Quality Software Management, Volume 3 (Dorset House Publishing, 1994), and V. Satir, The New People Making (Science and Behavior Books, 1988).

5 http://www.robertfrost.org/indexgood.html

6 J. Blumen, "The Prisoner's Dilemma in Software Development." Ethical Spectacle, Vol. I, No. 9, September, 1995.

7 B.A. Lieberman, "UML Activity Diagrams: Versatile Roadmaps for Understanding System Behavior." The Rational Edge, April 2001 (http://www.therationaledge.com/content/apr_01/t_uml_bl.html)

8 For more information on prioritization and project scope, see Chapter 20 of D. Leffingwell and D. Widrig, Managing Software Requirements. Addison-Wesley, 2000.

9 A complementary approach to this technique is outlined in an article by Walker Royce in the May 2001 issue of The Rational Edge, in which he suggests conducting a cost/value analysis for added features based on cost of development and associated customer perceived value. (http://www.therationaledge.com/content/may_01/f_sdev2_wr.html)

For more information on the products or services discussed in this

article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Book Review

The Leadership Engine: How Winning Companies Build Leaders at Every Level by Noel M. Tichy

HarperBusiness, 1997

Cover Price: US$27ISBN: 0-88730-793-0(384 Pages)

Like many readers, I have grown somewhat skeptical regarding the plethora of management and leadership books on the market today. How much value can there be in all this personal philosophy? Seems like everyone wants to be a guru in some form or fashion.

That's why I was so pleasantly surprised by this contribution from Noel Tichy.

In thinking about this review, I tried to ascertain what makes his book different from the others. I believe Tichy captures not only the "big picture" on leadership best practices, but also the tactical implementations required to achieve larger goals. At the highest level, we all know that leadership requires vision, communication, commitment, and so forth. But at the next level down, in the trenches, what does leadership provide and what does it look like? Tichy fills in a lot of the blanks left unaddressed by sweeping pronouncements. For example, lots of "experts" have told us that leaders need to develop the people they're leading in order to be truly successful, but they have not (as Tichy has done) delved into the importance of having a "teachable point of view" -- an ability to draw from one's own experiences and relate them to others.

Tichy lays out six principles for his "Leadership Engine": Teaching, Learning, Ideas, Values, Energy, and Edge. Edge, he explains, is a leader's ability to face reality and make tough decisions, even if the end result is that the leader must step down for the good of the organization. (How many corporate leaders do you know who would do this?) In Tichy's view, the best leaders are "enablers" rather than "doers." They work their initiatives through other people rather than doing it all themselves. They can only accomplish this, he adds, if they develop people sufficiently to ensure that proper execution can occur at all levels. The book approaches this challenge from a very practical perspective, suggesting specific things managers can do in order to become more effective leaders.

Here are Tichy's main assertions:

● Leaders operate on three levels that reinforce each other: Technical, Cultural, and Political. (This multi-level approach is critical for leaders to be truly effective.)

● Performance and behavior matter, not just performance. (Leaders can achieve desired results only by being both a good performer and a good employee -- and the two are not necessarily the same.)

● Leaders must prepare their organizations to win in the short term and be stronger for the long term. (Although senior executives often cut jobs and expenses to achieve short-term success, true leaders understand that this is not a formula for long-term growth and prosperity.)

● The legacy of winning leaders lies in creating other winning leaders. (This will sustain long term growth throughout the organization.)

● Good athletes don't always make good coaches; good leaders must have the ability to teach what they know. (We often take this ability for granted, but not everyone has it. When they move to or try to sustain a leadership role, managers must invest in their people and be satisfied with receiving recognition based on their success.)

The main theme here is that effective leaders teach leadership and are continually building leaders at all levels. Tichy provides many insightful examples as he explores this concept (although he is also long-winded and repetitive at times), often returning to Jack Welch, former CEO and Chairman of General Electric, as a reference point. Mr. Welch has been cited by many journalists, business experts, and academics as the CEO of the Century, and has been a significant proponent of building leaders throughout GE. As he examines successful leaders' formative years and draws linkages to their current methods and perspectives, Tichy also reminds us that developing good leadership skills is a lifelong endeavor and one that benefits from exposure to a wide variety of experiences.

- Sid FuchsRational Software Corporation

Read a review of Programming Web Services with XML-RPC by Simon St. Laurent, Joe Johnston, and Edd Dumbill.

Copyright Rational Software 2001 | Privacy/Legal Information

Book Review

Programming Web Services with XML-RPC by Simon St. Laurent, Joe Johnston, and Edd Dumbill

O'Reilly & Associates, 2001

Cover Price US$34.95ISBN 0-596-00119-3(230 Pages)

Before I opened this book, I knew I was interested in the Web Services and XML (Extensible Markup Language) part of the title, but I was not so sure about the RPC part. After all, the Web Services technology everyone's been talking about lately is SOAP (Simple Object Access Protocol). Now that I have read the book, however, I understand:

● The main differences between XML-RPC (Remote Procedure Call) and SOAP (SOAP is object-oriented and allows other transport layers besides HTTP; RPC is simpler and stateless).

● When to use XML-RPC and SOAP, respectively.

● How to use XML-RPC.

● More about the origins of SOAP within RPC technology.

● The key to XML-RPC technology in the HTTP Post request.

For all of these reasons, and for the variety of applications it discusses, the book is worthwhile reading. An advanced book on SOAP would complement this offering very nicely.

This book concentrates on the XML-RPC protocol specification and its uses in Web Services, including implementation in Java, Perl, PHP (hypertext preprocessor) server-side scripting language, Python, and Active Server Pages so that these languages can talk to each other. In XML-RPC, the XML markup language describes Remote Procedure Calls (RPC) transmitted via HTTP. The book's Foreword includes a brief history of this technology, written by one of its inventors. There's also an interesting Appendix on the XML and the HTTP used in the text.

It turns out that Web Services that rely on protocols such as XML-RPC facilitate interoperability between computer programs in different languages running on different operating systems -- and this aids in

building and debugging distributed applications. Basically, it works like this: An XML-RPC client specifies a method name and parameters packaged in an XML file payload and issues a Post request to a target server where the method is called. (The book's Appendix reminded me that forms can post/upload files by specifying an input of type file and a content type of multipart/form-data; for efficiency, XML-RPC headers specify a content type [MIME type] of text/xml instead.) The method's output is sent as an XML packaged response, including XML structs converted from the host programming language data type notation. The client need not be a browser. In contrast to Java's Remote Method Invocation, which supports CORBA interoperability, XML-RPC does not pass objects, and is therefore cheaper to implement.

The authors discuss both the advantages of open programming interfaces and the scalability and security minefields involved in using XML-RPC. In the process they document many examples, using a variety of languages and libraries:

● The helma.xmlrpc library for RPC-XML clients and servers for Java.

● The Frontier::RPC library for Perl.

● The xmlrpc.inc and xmlrpcs.inc library for PHP.

● Pythonware's xmlrpclib for Python.

● xmlrpc.asp.

● A C++ COM object implementation of XML-RPC for Active Server Pages.

Depending on which of these languages you have available, you may want to experiment with some examples and gain insights from reading the others. One example application used PHP to integrate two Web applications (a discussion server and a news database) into one interface, so that users could annotate technology news articles and read comments posted by others. The Active Server Pages example used XML-RPC to make an MSAccess address book accessible to a Linux Web server on the network and a MySQL address book on Linux available to an IIS server on Windows NT. Before reading this book, I had worked with calling, receiving, and applying XSL (Extensible Style Language) stylesheets to the responses of some webcrawlers and public Web Services allowing Web clients. The examples in the book helped me to better understand the internals and the variety of application types for which XSL may be used.

I recommend this book for Web developers and program integrators who want to use the proven, and relatively simple, XML-RPC standard to integrate systems or to provide services, or who want an excellent foundation for moving on to more complex protocols such as SOAP.

- Evelyn RomanRational Software Internet Services

Read a review of The Leadership Engine: How Winning Companies

Build Leaders at Every Level by Noel M. Tichy.

Copyright Rational Software 2001 | Privacy/Legal Information

Points of Integration Between Rational RequisitePro and Rational Rose

by Brenda Cammarano Senior Technical Evangelist

Rational Suite

The first installment of this series in the July issue of The Rational Edge provided an overview of the points of integration in the Rational Suites and explained how to set up projects with Rational Administrator. Now, in this brief article -- as well as several more to follow in upcoming issues -- I will address specific points of integration between two specific Rational Suite® products and explain how users can leverage them to best advantage. This month we'll focus on integrations between Rational Rose® and Rational RequisitePro® in Rational Suite v2001A.

Based on the Unified Modeling Language (UML), Rational Rose is the industry's leading model-driven development tool; Rational RequisitePro is the easiest and most intuitive tool for managing requirements. Together, they give developers and designers the tightest and most robust integration available between a model-driven development tool and a requirements management tool. We call this combined capability Integrated Use-Case Management, or IUC for short. IUC is available to all customers who purchase Rational Rose and Rational RequisitePro.

With IUC, you can track the evolution of functional software requirements expressed in use-case flows of events, and ensure that these flows validate the high-level requirements your software team is tasked to implement. Developers can continue to access use-case documents created in Microsoft Word from Rational Rose, but with the added benefit that those documents are now tracked for changes in Rational RequisitePro.

IUC lets you maintain all use-case properties as a single "unit" that spans your design and requirements management tools. The graphical properties

of use cases are expressed as use-case diagrams in Rational Rose; their textual properties (flows of events, priorities, etc.) are stored as documents and requirements attributes in Rational RequisitePro. This means developers and designers can instantly access use-case specifications, attributes, and traceability information from Rational Rose. Similarly, analysts can access use-case diagrams -- which provide the "big picture" for the system -- from RequisitePro.

The integration automatically maintains synchronicity between use-case names in Rational Rose and the text of their associated use-case requirements in Rational RequisitePro. Whenever there is a change in either the use-case name (in Rose) or the text of the associated requirement (in RequisitePro), the integration automatically updates both applications with the latest change.

How do you set up the integration between Rational Rose and Rational RequisitePro? By default, the integration uses whatever Rose model and RequisitePro project you specify in the Rational Suite project you create in Rational Administrator. For any given Rational Suite project, Rational Administrator allows you to set up a single RequisitePro project and multiple Rose models. If you want to link a Rose model to multiple RequisitePro projects, then you can manually browse to an alternative RequisitePro project from your Rose Model -- specific steps are described below under Establishing the Integration (#2). Similarly, from RequisitePro you can specify Rose models that are different from the ones you specified in the Rational Suite project.

Integration Capabilities and Benefits

What else can you do with IUC? Let's look at the capabilities.

From Rational Rose, you can:

● Create and store use-case documents as Rational RequisitePro documents, which are tracked for changes and link to other requirements documents. You can also use proven use-case templates from RequisitePro to create better use-case documents.

● Associate a use case with an existing RequisitePro requirement.

● Create a set of functional software requirements from a use case (to track use-case changes).

● Edit and view requirement attributes and traceability links for the functional software requirements expressed in a use-case flow of events.

● Navigate quickly from use-case diagrams to use-case specification documents in RequisitePro.

From Rational RequisitePro you can:

● Create a use case in Rose from a use case in RequisitePro.

● Navigate from a use case in RequisitePro to its associated use-case

diagram in Rose.

These capabilities bring some big benefits. The ability to manage requirements in a use case flow of events is critical to tracking and understanding the state of your project; it helps ensure that you're solving the right problem and developing the right solution. In addition, IUC provides requirements managers with more accurate and timely information -- from within their own working environment.

Establishing the Integration

Now that you're convinced of the rewards you'll reap from establishing the integration between Rational Rose and Rational RequisitePro, just follow the steps below.

1. Create Association from Rational Rose. (Note: This step is necessary only if you choose not to use the default association created by your Rational Suite project.)a) Associate a RequisitePro project to an open Rose model through Tools -> RequisitePro -> Associate model to RequisitePro Project

And optionally

b) Override the model-level association with a package-level association. You can associate a different RequisitePro project with each Rose package by right-clicking the package and selecting RequisitePro -> Associate Package with Project

This allows you to select different use-case document templates for different use cases in Rose. The association will affect only those use cases created in that package after the association is established.

2. Create Association from Rational RequisitePro.

From a RequisitePro view or document, select a requirement. Then, in the menu select RequisitePro -> Requirement -> Rational Rose -> Associate to Use Case

Figure 1: Requirement Use Case Association Dialog

In the Requirement Use Case Association dialog (Figure 1), you can then click Browse to bring up the dialog in Figure 2. This lets you select a different Rational Rose model to associate with the requirement.

Figure 2: Rational Rose Model Selection Dialog

When you find the model you want to associate with the requirement, you can select it and click Open to return to the dialog in Figure 1. You'll see the file name of the model you've just chosen in the Rose Model File box, and the tree below that will display all the items in the model. Now you can find the use case you want to associate with the requirement (or create a new use), select it, and complete the association.

Note that Use-Case Brief Descriptions can be entered either in the Rose Documentation field of the Use Case Specification dialog -- which you can

access by right clicking the use case in the Rose Browser -- or in the RequisitePro Use-Case Properties dialog -- which you can access by going to the Open Properties and Documents dialog, selecting the document, and then clicking on Properties. At this time, modifications made through either of these two entry points must be synchronized manually. I recommend that you select one tool to enter the Brief Description, then copy and paste to the other tool, so that the Brief Descriptions will be identical.

Want to Know More?

For more detailed information on this integration, see the following whitepapers at www.rational.com/products/astudio/whitepapers.jsp:

● "Applying Requirements with Use Cases"

● "Use-Case Management with Rational Rose and Rational RequisitePro"

Acknowledgements

Many thanks to Catherine Connor, Senior Technical Marketing Engineer for Rational RequisitePro, for her help and advice on this article.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Capturing Architectural Requirements

by Peter EelesRational Services Organization, UK

Capturing requirements is difficult. Capturing architecturally significant requirements is particularly difficult. This article discusses the root causes of this difficulty, and suggests a systematic approach to capturing architectural requirements to ensure that these elusive, and yet extremely important, system specifications are not overlooked.

What Is an Architectural Requirement?

Because this article focuses on an approach to gathering requirements of particular significance to the architecture of a system1, let's start with the definition of an architectural requirement.

The Rational Unified Process® (RUP®) gives the following definition for any requirement:

A requirement describes a condition or capability to which a system must conform; either derived directly from user needs, or stated in a contract, standard, specification, or other formally imposed document.

An architectural requirement, in turn, is any requirement that is architecturally significant, whether this significance be implicit or explicit. Implicit architectural requirements are those requirements that have particular attributes. For example, any high-risk, high-priority, or low-stability requirement could be considered to be architecturally significant. However, this article will focus primarily on explicit requirements, which are often technical in nature. The following are examples of explicit architectural requirements:

● The product will be localized (support multiple human languages).

● The persistence will be handled by a relational database.

● The database will be Oracle 8i.

● The system will run seven days a week, twenty-four hours per day.

● An online help system is required.

● All presentation logic will be written in Visual Basic.

As you may notice, these requirements are extremely mixed. Some are functional, some non-functional; some are independent of technical mechanisms, others are not. What we need is a systematic approach that provides a framework for classifying architectural requirements, which ensures that valuable statements such as those listed above are not overlooked.

The FURPS+ System for Classifying Requirements

One such classification system was devised by Robert Grady at Hewlett-Packard.2 It goes by the acronym FURPS+ which represents:

● Functionality

● Usability

● Reliability

● Performance

● Supportability

The "+" in FURPS+ also helps us to remember concerns such as:

● Design requirements

● Implementation requirements

● Interface requirements

● Physical requirements

Let's look at each category in detail.

Functional Requirements

These requirements generally represent the main product features. In a warehouse application, we might have requirements pertaining to order processing or stock control, for example. However, functional requirements are not always domain-specific. Providing printing capability is a functional requirement of particular significance to architecture, for example.

Table 1 lists additional functional requirements that might be considered.

Table 1: Architecturally Significant Functional Requirements

Function Description

Auditing Provide audit trails of system execution.

Licensing Provide services for tracking, acquiring, installing, and monitoring license usage.

Localization Provide facilities for supporting multiple human languages.

Mail Provide services that allow applications to send and receive mail.

Online help Provide online help capability.

Printing Provide facilities for printing.

Reporting Provide reporting facilities.

Security Provide services to protect access to certain resources or information.

System management Provide services that facilitate management of applications in a distributed environment.

Workflow Provide support for moving documents and other work items, including review and approval cycles.

Usability, Reliability, Performance, and Supportability Requirements

The remaining "URPS" categories describe non-functional requirements that are generally architecturally significant.

● Usability is concerned with characteristics such as aesthetics and consistency in the user interface.

● Reliability is concerned with characteristics such as availability (the amount of system "up time"), accuracy of system calculations, and the system's ability to recover from failure.

● Performance is concerned with characteristics such as throughput, response time, recovery time, start-up time, and shutdown time.

● Supportability is concerned with characteristics such as testability, adaptability, maintainability, compatibility, configurability, installability, scalability, and localizability.

Design, Implementation, Interface, and Physical Requirements

The "+" in the FURPS+ acronym is used to identify additional categories that generally represent constraints.

● A design requirement, often called a design constraint, specifies or constrains the options for designing a system. For example, if you specify that a relational database is required, that's a design

constraint.

● An implementation requirement specifies or constrains the coding or construction of a system. Examples might include required standards, implementation languages, and resource limits.

● An interface requirement specifies an external item with which a system must interact, or constraints on formats or other factors used within such an interaction.

● A physical requirement specifies a physical constraint imposed on the hardware used to house the system -- shape, size, or weight, for example.

Realizing Requirements

From the descriptions above, we can easily see that some functional requirements, and most requirements in the other FURPS+ categories, are architecturally significant. Now let's look at how we might classify the seemingly unrelated architectural requirements we listed earlier. Using the FURPS+ classification we can see that:

● "The product will be localized (support multiple human languages)" is a supportability requirement.

● "The persistence will be handled by a relational database" is a design requirement.

● "The database will be Oracle 8i" is an implementation requirement.

● "The system will run seven days a week, twenty-four hours per day" is a reliability requirement.

● "An online help system is required" is a functional requirement.

● "All presentation logic will be written in Visual Basic" is an implementation requirement.

Knowing how such requirements are realized will help us ask the right questions of our stakeholders. There is also value in understanding the relationships between categories of requirement that, at first glance, appear very disparate. Considering architectural mechanisms can assist us on both counts.

Architectural Mechanisms

In simple terms, an architectural mechanism represents a common solution to a frequently encountered problem. Architectural mechanisms are often used to realize architectural requirements. Table 2 shows three categories of architectural mechanisms and shows how architectural mechanisms are expressed in each of these categories.

Table 2: Three Categories of Architectural Mechanisms

Analysis Mechanism Design Mechanism

Implementation Mechanism

PersistenceRDBMS

Oracle

Ingres

OODBMS ObjectStore

Communication

Object request broker

Orbix

VisiBroker

Message queueMSMQ

MQSeries

An analysis mechanism represents an implementation-independent solution. Table 2 shows two analysis mechanisms: persistence and communication. For more examples of analysis mechanisms, see the Analysis Mechanism Summary in Appendix A.

A design mechanism is a refinement of an analysis mechanism. It assumes some details of the implementation environment but is not tied to a specific implementation. In our example, the communication analysis mechanism may be realized as a design mechanism such as an object request broker or a message queue.

Finally, an implementation mechanism is a refinement of a design mechanism, and specifies the exact implementation of the mechanism. In our example, an object request broker may be implemented using either Orbix or VisiBroker.

Figure 1 summarizes the relationship between requirements and mechanisms, showing refinements of the FURPS requirements, design requirements and implementation requirements as well as architectural mechanisms at different levels of refinement.

Figure 1: Relationship Between Requirements and Mechanisms

The Challenge of Gathering Architectural Requirements

Before we discuss a systematic approach to gathering architectural requirements, it is worth noting why such an approach is needed. In essence, this is because many of the FURPS+ requirements we mentioned earlier are relevant in a system-wide context and help drive the design of the foundations (i.e., the architecture) of the system we're building. In fact, on some projects, architectural requirements can be significantly more important than their domain-specific counterparts. If you were designing a life-support machine, for example, then availability ("up time") would be pretty high on your list.

So why is it that architectural requirements are often overlooked? Because they are difficult to gather -- and that's why being "systematic" about how these requirements are gathered can make a real difference.

Gathering architectural requirements means venturing into uncharted territory (in contrast to gathering more domain-specific requirements) for a number of reasons:

● In most (although not all) systems, from an end-user perspective, domain-specific requirements are more visible than their architectural counterparts. Consequently, emphasis is placed on gathering these domain-specific requirements because they are perceived as the most valuable.

● Stakeholders are not usually familiar with the majority of architectural requirements. Although they are comfortable with specifying domain-specific features such as "Order Processing" and "Stock Control," they are less familiar with "Reliability" and "Usability," and think of them as technical issues that lie outside their area of concern.

● Systematic techniques for gathering architectural requirements are generally less well known than techniques for gathering domain-specific requirements.

Using a systematic approach can help overcome these difficulties, as we shall see below.

An Approach for Gathering Architectural Requirements

The approach to gathering architectural requirements we will explore is simple:

1. Maintain a complete list of architectural requirements (regardless of whether all items are relevant to a particular project). See Appendix B: Architectural Requirements for a sample list.

2. For each architectural requirement, formulate one or more questions that can help in the specification process. Make sure all the system's stakeholders can understand these questions.

3. Assist stakeholders by showing them the potential impact of

answering a question one way or the other.

4. Capture the responses from your stakeholders to each of the questions.

5. Assist the architect by ensuring that the stakeholders -- in addition to answering these questions -- assign a priority or weighting to each architectural requirement. This weighting will allow the architect to make tradeoffs between requirements.

It is worth noting that this approach is possible because, at a high level, the set of architectural requirements that must be considered is finite. You can also apply this approach to requirements gathering within particular problem domains that also have finite, well-defined, sets of considerations. For a financial system, for example, there would be an imperative to pose certain finance-related questions.

The Architectural Requirements Questionnaire

This approach is best represented in the form of a simple table provided to stakeholders as an Architectural Requirements Questionnaire. Table 3 shows a portion of such a questionnaire and includes example answers. For a complete Architectural Requirements Questionnaire template, see Appendix C.

Table 3: Portion of an Architectural Requirements Questionnaire

Requirement Questions Impact Answers Priority

Licensing Will the system, or parts of the system, be licensed?

Are there any constraints on the mechanism used to provide licensing capability?

The greater the sophistication of the licensing mechanism, the longer the time to market, and the greater the long-term maintenance cost.

The stock control module will be marketed as a separate component of the system and will require licensing. The FlexLM tool is used throughout our organization to provide licensing capability.

Medium

Availability Are there any requirements regarding system "up time"? This may be specified in terms of Mean Time Between Failures (MTBF).

The higher the availability, the longer the time to market.

Availability is a key product feature. The product must have an MTBF of 60 days.

High

Platform support

What platforms must the system support?

Development for a single platform shortens the time to market. It also allows closer integration with platform features.

Development for multiple platforms lengthens the time to market. Close integration with platform features is lessened, increasing the maintenance of the system.

The product will be released on the following UNIX platforms:

Sun SolarisIBM AIXHPUX

High

Note that this questionnaire is used in the Elicit Stakeholder Requests activity in the Rational Unified Process (RUP). Once completed, the questionnaire is then used in the definition of some of the most important artifacts used in the development process, including the Use-Case Model and Supplementary Specification (which together provide a formal specification of system requirements).

Rational RequisitePro® can be of great help in relation to the questionnaire:

● It allows you to create multiple "views" of the questionnaire. This is particularly valuable if you are interviewing different stakeholders because you can filter by role. If you are interviewing a marketing person, for example, you might want his or her responses to a certain subset of questions but not to complex technical issues. Rational RequisitePro allows you to assign a "role" attribute to each question, which makes information gathering more efficient.

● It gives you traceability between architectural stakeholder requests and both use-case requirements and supplementary requirements. You can specify this traceability through links, and RequisitePro provides traceabilty matrices to help you visualize these links.

Avoiding Common Pitfalls

When gathering any requirements -- not just architectural requirements -- there are a number of potential pitfalls. This section discusses these pitfalls and provides suggestions for avoiding them.

The "Shopping Cart" Mentality. The technique described in this article has been used on more than twenty projects to date, and every single one of these projects has suffered from stakeholders' false impression that specifying requirements is like filling up a shopping cart. In the absence of any other criteria, a requirements gathering effort can amount to a futile exchange along the following lines:

Analyst: Does the product need to be localized?

Stakeholder: That sounds good. We should plan to address foreign markets.

Analyst: And what about security?

Stakeholder: Oh yes, the product should be secure.

Analyst: Tell me about your reliability expectations.

Stakeholder: Definitely twenty-four by seven -- no down time. That's what we need to beat our competitors.

And so on. If you could offer your stakeholders a solution that solved the worldwide shortage of dilithium crystals3, they'd want it. But every desired requirement comes at a price; stakeholders can't just put everything they want into a shopping cart at no cost. Be careful not to fall into the trap of presenting them with the equivalent of a shopping list from which they pick and choose.

The trick is to ensure that your stakeholders understand the cost of their purchases, which is why an impact statement is associated with each question in the questionnaire.

The "This Is Too Technical for Me" Attitude. Some stakeholders may dismiss the Architectural Requirements Questionnaire as a technical document whose content lies outside their area of concern. Often, the reason for this perception is that the questions don't discuss their familiar domain concepts, so they treat the document as less important than techniques (such as use-case modeling) for capturing more visible, domain-specific requirements.

Once again, it's important to ensure that your stakeholders understand the value of taking time to answer questions your questionnaire poses. It's often easiest to demonstrate this value by giving examples of problems that arise in the absence of a questionnaire!

The "All Requirements Are Equal" Fallacy. Another common pitfall stems from giving all requirements the same priority. Without exception, projects that fall into this trap classify all requirements as high priority.

Architectural requirements must be prioritized to indicate to the architect -- or anyone else -- which are the most important requirements for the finished system. No design tradeoffs can be made if all requirements are assigned the same priority. If you start to get bogged down when prioritizing requirements, try considering them two at a time. The most important requirements should naturally "bubble up" to the top of the list.

The "Park It in the Lot" Problem. In some organizations, stakeholders dutifully collect requirements because an analyst told them to and prioritize them (again, because an analyst told them to). And then, for some reason, these requirements are then placed "on the shelf," never to be used again. It's critical to ensure that your stakeholders understand the

value of capturing architectural requirements and that these requirements are applied throughout the development of the system.

The "Requirements That Can't Be Measured" Syndrome. This can be a pitfall for both domain-specific and architectural requirements. It's important to ensure that all stated requirements are both unambiguous and measurable. "The system will have a state-of-the-art interface," for example, does not meet these criteria, because the definition of state-of-the-art is highly subjective.

The "Not Enough Time" Complaint. Specifying architectural requirements is a complex task that can't be done quickly. If stakeholders balk at scheduling adequate time for the activity or grow impatient during the process, remind them how important these requirements are in building a foundation for the system, and don't allow them to short-circuit your efforts.

The "Lack of Ownership" Problem. It is critical for the system analyst and software architect to work collaboratively to create the Architectural Requirements Questionnaire, and for the analyst to fully understand the content. After all, the analyst will be gathering the requirements. If he or she conveys the impression that the questionnaire is too technical to comprehend and cannot provide clarification when necessary, that will only reinforce stakeholders' misperceptions that the questionnaire is of little use.

The "All Stakeholders Are Alike" Misconception. Many different stakeholders have input during the requirements gathering process, and you need to address the right questions to the right people. For example, product management is probably the group to ask about specifying the ability to license elements of the system, but requirements for system usability are best specified by end users. Before you talk with stakeholders, take the time to identify which group is responsible for answering which questions.

The "Too General" Tendency. The Architectural Requirements Questionnaire should be treated as an elicitation technique similar to interviewing, brainstorming, and so on. The only difference between this technique and others is that the primary focus is on capturing architectural requirements for the system. Architectural requirements are often specified at a very general level. For example, a requirement might state that the response time of any transaction is less than three seconds. It's likely, however, that although response times for specific transactions must conform to this particular requirement, some transactions will take significantly longer. If you don't make requirements as specific as possible, then elements of your system are likely to be over-engineered, to comply with architectural requirements that may not apply. Once again, the RUP is helpful here. It provides three possible locations for architectural requirements, which reflect three different levels of specificity, as shown in Table 4.

Table 4: Levels of Specificity and RUP Locations for Architectural Requirements

Relates to: Example Location in a RUP Artifact

A particular event flow within a use case (most specific)

If, in the basic flow, the plane undercarriage fails to engage, then an alarm will be sent to the central monitoring station in less than one second.

"Flow of events" section in a use-case specification.

An entire use case

Any order that is processed must scale to support more than 10,000 items.

"Special requirements" section in a use-case specification.

The entire system (most general)

Due to the nature of our target markets, the system must be deployed in English, French, Chinese, and Arabic.

Supplementary specification.

Where to Go from Here

As we've seen, capturing architecturally significant requirements is difficult and fraught with pitfalls. But help is available. Use a tried and true classification scheme for architectural requirements and an elicitation technique to ensure that these requirements are not overlooked. Use the appendices in this article, which contain a Sample Architectural Requirements Questionnaire, and summaries of architectural analysis mechanisms, and architectural requirements.

You can also take advantage of the best practices and workflow capabilities built into the Rational Unified Process and Rational RequisitePro. These tools are based on the experiences of many software professionals just like yourself who have been down the same challenging path, so using them will get you off to a great start and sustain you on the journey, too.

Acknowledgments

I'd like to thank a number of my colleagues at Rational for reviewing the content of this article and providing valuable feedback, including Dave Brown, Maria Ericsson, Kelli Houston, Volker Kopetzky, and Wojtek Kozaczynski.

Appendices

Appendix A: Analysis Mechanisms

Appendix B: Architectural Requirements

Appendix C: Sample Architectural Requirements Questionnaire

Notes

1 Much of the advice in this article can be applied to requirements gathering in general.

2 As presented in Robert Grady, Practical Software Metrics for Project Management and Process Improvement. Prentice-Hall, 1992.

3 As every Trekkie knows, dilithium can work in conjunction with deuterium to power both warp drives and weaponry. Unfortunately, it is native only to planets heretofore unvisited by Earthlings.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Appendix A: Analysis Mechanisms

From "Capturing Architectural Requirements" by Peter Eeles

Consider the following analysis mechanisms when gathering architectural requirements.

Analysis Mechanism Description

Auditing Provides audit trails of system execution.

Communication Allows distributed processes to communicate.

Debugging Provides elements to support application debugging.

Error Management Allows errors to be detected, propagated, and reported.

Event Management Supports the use of asynchronous messages within a system.

File Management Provides services for accessing a file system.

Graphics Supports user interface services, such as 3D rendering.

Information Exchange Supports data format conversion.

Licensing Provides services for acquiring, installing, tracking, and monitoring license usage.

Localization Provides facilities for supporting multiple human languages.

Mail Services that allow applications to send and receive mail.

Mega-data Support for handling large amounts of data in a client-server environment.

Memory Management Services for abstracting how memory is allocated and freed.

Meta-data Supports the runtime introspection of components.

Online help Provides online help capability.

Persistence Services to manipulate persistent data.

Printing Provides facilities for printing.

Process Management Provides support for the management of processes and threads.

Reporting Provides reporting facilities.

Resource Management Provides support for the management of expensive resources, such as database connections.

Scheduling Provides scheduling capability.

Security Provides services to protect access to certain resources or information.

System Management Services that facilitate management of applications in a distributed environment.

Time Services to synchronize time on a network, and to translate times into different time zones.

Transaction Management A mechanism for handling ACID1 transactions.

Workflow Support for the movement of documents and other items of work, typically through an organization.

1 The acronym ACID refers to four characteristics of a transaction: Atomic, Consistent, Isolated and Durable.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Appendix B: Architectural Requirements

From "Capturing Architectural Requirements" by Peter Eeles

This appendix summarizes the architectural requirements to consider.

Functionality

Functional requirements that are of particular relevance to architecture are generally derived from the analysis mechanisms listed in Appendix A.

Usability

Requirement Description

Accessibility The ease with which different facets of the system are exercised.

Aesthetics The aesthetic quality of the user interface.

Consistency The consistent use of mechanisms employed in the user interface. This applies both within the system, and with other systems.

Reliability

Requirement Description

Accuracy The accuracy of any calculations performed.

Availability1 The amount of system "up time."

Recoverability The elegance with which the system recovers from failure.

Performance

Requirement Description

Recovery time The time to recover from a system failure.

Response time The time for the system to provide a response.

Shutdown time The time for the system to shut down.

Start-up time The time for the system to start up.

Throughput The capacity of the system to support a given flow of information.

Supportability

Requirement Description

Adaptability2 The ease with which the system is adapted to new environments.

Auditability The ease with which the system provides audit trails of its execution.

Compatibility The compatibility of this system with previous versions of the system.

Configurability3 The ease with which the system is configured.

Installability The ease with which the system is installed.

Localizability The level to which the system supports multiple human languages.

Maintainability4 The ease with which the system is maintained.

Scalability The ease with which the system can scale in terms of data volumes and users.

Testability The ease with which the system is tested.

Design Requirement

A design requirement is simply a constraint on the refinement of an analysis mechanism, such as those listed in Appendix A. For example, a requirement for a relational database is a constraint on the refinement of a persistence analysis mechanism. The set of design requirements to consider is therefore derived directly from the list of analysis mechanisms.

Implementation Requirement

Requirement Description

Third-party components Any constraints on the use and cost of third-party components.

Implementation languages The implementation languages to be used.

Platform support The platforms that the system will support.

Resource limits Limits on the use of resources such as memory and hard disk space.

Standards compliance Any standards to which the system must conform.

Interface Requirement

Requirement Description

External systems External systems with which this system must interface.

Interface formats The format of any data passed between this system and external systems.

Physical Requirement

Requirement Description

Shape The shape of the resulting hardware housing the system.

Size The size of the hardware housing the system.

Weight The weight of the hardware housing the system.

Footnotes

1 Availability is also known as robustness.

2 Adaptability is also known as evolvability, plugability and upgradeability.

3 Configurability is also known as customizability, extensibility and flexibility.

4 Maintainability is also known as serviceability.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Appendix C: Sample Architectural Requirements Questionnaire

From "Capturing Architectural Requirements" by Peter Eeles

This section contains a sample Architectural Requirements Questionnaire. The examples shown here exclude the answer and priority columns, which are discussed in the main body of my article.

This questionnaire also groups together related items that were previously categorized in separate sections. Consider all aspects of auditing, for example. Auditing manifests itself as a functionality requirement (Is audit capability required?), supportability requirement (What level of auditing is needed?) and design requirement (Are there any design constraints on the mechanism used to provide audit capability?). Such requirements tend to be related to particular analysis mechanisms and are therefore grouped together in the "Analysis Mechanisms" section.

Analysis Mechanisms

Requirement Questions Impact

Auditing

Functionality

Supportability

Design requirement

Is audit capability required?

What level of auditing is needed?

Are there any constraints on the mechanism used to provide audit capability?

The greater the sophistication of the audit mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Authentication

Functionality

Design requirement

Is there any requirement for authentication?

Are there any constraints on the mechanism used to provide authentication capability?

The greater the sophistication of the authentication mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Communication

Functionality

Design requirement

Will inter-process communication be required?

Are there any constraints on the mechanism used to provide communication capability?

The greater the sophistication of the communication mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Error management

Design requirement

Are there any constraints on the mechanism used to provide error management capability?

The greater the sophistication of the error management mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Event management

Design requirement

Are there any constraints on the mechanism used to provide event management capability?

The greater the sophistication of the event management mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Licensing

Functionality

Design requirement

Will the system, or parts of the system, be licensed?

Are there any constraints on the mechanism used to provide licensing capability?

The greater the sophistication of the licensing mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Localization

Functionality

Supportability

Design requirement

Does the system need to support multiple human languages?

Which human languages need to be supported?

Are there any constraints on the mechanism used to provide localization capability?

The greater the sophistication of the localization mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Online help

Functionality

Design requirement

Will online help be required?

Are there any constraints on the mechanism used to provide online help?

The greater the sophistication of the online help mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Persistence

Functionality

Design requirement

Will data manipulated by the system be written to disk? Are both shared persistent data and user-specific persistent data required? Will shared persistent data need to be made available to a "traveling" user who is working offline?

Are there any constraints on the mechanism used to provide persistence?

The greater the sophistication of the persistence mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Printing

Functionality

Design requirement

Is printing capability required?

Are there any constraints on the mechanism used to provide printing capability?

The greater the sophistication of the printing mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Reporting

Functionality

Design requirement

Is reporting capability required?

Are there any constraints on the mechanism used to provide reporting capability?

The greater the sophistication of the reporting mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Scheduling

Functionality

Design requirement

Will certain system actions need to be scheduled?

Are there any constraints on the mechanism used to provide scheduling capability?

The greater the sophistication of the scheduling mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Security

Functionality

Design requirement

Will elements of the system need to be secure?

Are there any constraints on the mechanism used to provide security capability?

The greater the sophistication of the security mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Transaction management

Functionality

Design requirement

Will transactional capability be required?

Are there any constraints on the mechanism used to provide transactional capability?

The greater the sophistication of the transaction management mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Workflow

Functionality

Design requirement

Will movement of organizational units of work be required?

Are there any constraints on the mechanism used to provide workflow capability?

The greater the sophistication of the workflow mechanism, the longer the time to market, and the greater the long-term maintenance cost.

Functionality

All functional requirements are addressed in the "Analysis Mechanisms" section above.

Usability

Requirement Questions ImpactAccessibility Are there any special requirements

that relate to the ease with which the system can be exercised (including by novice users)?

The greater the accessibility, the longer the time to market, and the greater the long-term maintenance cost.

Aesthetics Are there any special requirements regarding the "look and feel" of the system?

The greater the aesthetic appeal, the longer the time to market.

Consistency Are there any special requirements regarding the consistency of the user interface, both within the system and with other systems?

The greater the level of consistency, the longer the time to market.

Reliability

Requirement Questions ImpactAccuracy Are there particular constraints on

the accuracy of any calculations to be performed by the system?

The greater the level of accuracy, the longer the time to market.

Availability Are there any requirements regarding system "up time"? This may be specified in terms of Mean Time Between Failures.

The higher the availability, the longer the time to market.

Recoverability Are there any special requirements regarding recovery from a system failure?

The greater the level of recoverability, the longer the time to market.

Performance

Requirement Questions ImpactRecovery time Are there any requirements on the

time the system takes to recover from a system failure?

The shorter the recovery time, the more sophisticated the recovery mechanism and the longer the time to market.

Response time Are there any constraints on the time the system takes to respond to particular events, such as user interaction?

The shorter the response time, the longer the time to market.

Shutdown time Are there any requirements on the time the system takes to shut down?

The shorter the shutdown time, the more sophisticated the shutdown mechanism and the longer the time to market.

Start-up time Are there any requirements on the time it takes the system to start up?

The shorter the start-up time, the more sophisticated the start-up mechanism and the longer the time to market.

Throughput Are there any special requirements regarding throughput of data supported by the system?

The greater the throughput of data, the longer the time to market.

Supportability

Requirement Questions ImpactAdaptability Are there any special

requirements regarding adaptation of the software (including upgrading)?

The greater the system's adaptability, the longer the time to market, and the greater the long-term maintenance cost.

Compatibility Are there any requirements regarding this system and its compatibility with previous versions of this system or legacy systems providing the same capability?

The greater the compatibility, the longer the time to market.

Configurability Will the product be configured after it has been deployed? In what way will the system be configured?

The greater the configurability of the system, the longer the time to market, and the greater the long-term maintenance cost.

Installability Are there any special requirements regarding system installation?

The greater the installability of the system, the longer the time to market.

Maintainability Are there any special requirements regarding system maintenance?

The greater the maintainability of the system, the longer the time to market. The trade-off is between time to market of the first release of the system, and any subsequent releases.

Scalability What volumes of users and data will the system support? This may be expressed as a profile over time.

The greater the scalability of the system, the longer the time to market.

Testability Are there any special requirements regarding the testability of the system, such as testing "hooks" that allow integration with a testing tool?

The greater the testability of the system, the longer the time to market.

Design Requirement

All design requirements are addressed in the "Analysis Mechanisms" section above.

Implementation Requirement

Requirement Questions ImpactThird-party components Are there any constraints

(including cost) on the use of third-party components in the system?

How important is it for the system to be vendor independent, in terms of the third-party components it uses?

Are there any approved relevant alliances with third- party vendors?

The more third-party components used, the shorter the time to market. Also, the system is likely to be more maintainable because these components are effectively subcontracted.

Existing alliances typically remove the need for contractual negotiations, thereby improving time to market.

Implementation languages Are there any constraints on the programming languages used to develop the system?

Specification of particular programming languages must be reconciled with tool support and available skills. Time to market is increased if either tools or skills are unavailable.

Platform support What platforms must the system support? Which operating system versions must be supported?

Is Web access required? If so, then which Web browsers and versions must be supported?

Will the system be ported to new platforms in the future (over and above those explicitly stated for the system)?

Development for a single platform shortens the time to market. It also allows closer integration with platform features.

Development for multiple platforms lengthens the time to market. Close integration with platform features is lessened, increasing system maintenance. However, deployment on new platforms is quicker.

Resource limits Are there any limits on the use of system resources, such as memory or hard disk space?

The more constrained the available resources, the more efficient resource use is required. This increases time to market and lessens maintainability.

Standards compliance Are there any standards to which the system must conform? This may include coding standards or a user interface style guide.

Use of available standards can decrease time to market and improve consistency in a number of system areas.

Interface Requirement

Requirement Questions ImpactExternal systems Are there any external systems

with which this system must interface? Consider both provided and required interfaces.

The greater the sophistication of the interface to external systems, the longer the time to market, and the greater the long-term maintenance cost.

Interface formats Are there any constraints on the nature of the interface between this system and any external system, such as the format of data passed between these systems, and any particular protocol used?

The greater the sophistication of the format of interfaces, the longer the time to market, and the greater the long-term maintenance cost.

Physical Requirement

Requirement Questions ImpactShape Are there any constraints on the

shape of the hardware used to house the resulting system?

Constraints on the shape of the hardware may impose a specification of physical devices required by the system, such as storage devices or printers. The more constrained the hardware, the longer the time to market.

Size Are there any constraints on the size of the hardware used to house the resulting system?

Constraints on the size of the hardware may impose a specification of physical devices required by the system, such as storage devices or printers. The more constrained the hardware, the longer the time to market.

Weight Are there any constraints on the weight of the hardware used to house the resulting system?

Constraints on the weight of the hardware may impose a specification of physical devices required by the system, such as storage devices or printers. The more constrained the hardware, the longer the time to market.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Using Perl with Rational ClearCase Automation Library (CAL)

by Tom Milligan Software Engineering Specialist

Rational Services Organization

As a consultant for Rational, I spend a great deal of time working directly with Rational customers. Typically, I work together with Rational ClearCase® users to develop a customized solution to whatever unique challenge they are facing. Often in these encounters, I have found the sometimes overlooked ClearCase Automation Library (CAL) to be an especially handy tool for accessing and manipulating ClearCase data programmatically in a wide variety of situations.

Starting with ClearCase 4.0, both CAL and the CAL documentation are automatically installed with Rational ClearCase on Windows platforms. CAL provides a set of COM (Component Object Model) interfaces that are intended both as an integration platform and also as an API (Application Program Interface) that you can use to extend or customize Rational ClearCase. You can also use CAL to write scripts, stand-alone programs, or macros embedded in other applications. Because CAL is a COM-compliant library, you can use it from within any tool that can interface with COM, including Visual Basic, Visual C++, and Perl on Windows. Since COM is primarily a Windows-only technology, CAL is not available on UNIX platforms.

In this article, I'm going to talk about why you might want to use CAL, and some tools and resources that are useful for working with CAL in the Perl programming language. I'll also walk through a fairly simple Perl application that uses CAL as well as the Rational ClearQuest® API to generate a report combining ClearCase and ClearQuest data. An in-depth discussion of the Rational ClearQuest API is beyond the scope of this article, but it will be addressed in an upcoming edition of The Rational Edge.

A Very Quick Tech Review

In general, the concepts I'll discuss here are fairly straightforward and should be readily understandable by a technical audience. However, some basic familiarity -- or expertise -- in the following areas will help you get the most from this discussion:

● Software configuration management concepts in general and Rational ClearCase in particular. A quick review: Rational ClearCase is the market-leading tool for software asset management. Rational ClearQuest is a highly flexible defect and change tracking tool. Together, they enable Unified Change Management (UCM), Rational's out-of-the box process for managing change at the activity level.

● The Perl Programming Language. Perl is a remarkably popular programming language. Relatively easy to learn and often available for free, it has been described as the duct tape of not only the Internet, but also the entire computing world. If you're looking to learn Perl or just brush up on it, I recommend Learning Perl (the book with the llama on the cover) and for a more advanced reference Programming Perl1 (the book with the camel on the cover), both from O'Reilly & Associates. The latter title is included in the Perl CD Bookshelf -- see Useful Tools below.

● COM or Component Object Model. Developed by Microsoft, COM is a standard that defines a mechanism for software components to interact with each other. COM is an object-oriented technology, and its main building blocks are objects, interfaces, methods, and properties. There are numerous excellent resources and books on COM2, and even the more basic ones should give you enough information to start using CAL.

Why CAL?

You may already be familiar with another common method of accessing Rational ClearCase data from outside the Rational ClearCase user interface. The cleartool utility is a command-line interface that you can use to create, modify, and manage the information in Rational ClearCase VOBs (Versioned Object Bases) and views. The flexibility and ease of use of cleartool makes it an ideal solution for a wide range of problems. And in one sense, cleartool has always offered a kind of API to ClearCase, for both Windows and UNIX platforms. However, there are situations in which CAL offers a superior alternative to cleartool -- specifically when speed and performance are factors.

Certain languages are not well suited to invoking command-line tools and parsing the results; Visual Basic is a good example. Parsing results can be somewhat difficult in the C programming language as well. Perl, on the other hand, makes it easy to start command-line utilities and parse the results. But even when using Perl, you may want greater speed. Because CAL does not create a new process for each invocation, it can be considerably faster than a solution that uses cleartool's command-line interface. When a program invokes cleartool, the operating system must create a new process, and there is a significant amount of overhead associated with starting that new process -- allocating memory for it, creating a new entry in the process table, and so on.

I recently did some very basic performance measurements by timing a task that I completed with cleartool and then timing the same task using a CAL

implementation instead. For this specific operation, CAL was about 30 percent faster. While these results are empirical, and fairly rough, they show that CAL can offer a significant advantage when speed is important.

Useful Tools

Of course, CAL itself and the CAL documentation are essential for developing Perl scripts that access ClearCase data (without using cleartool). If you've installed Rational ClearCase, then you already have CAL and the documentation on your system.

Documentation for CAL includes conceptual material, diagrams, several examples, and reference pages for each interface in CAL. There are several other tools -- some essential, some merely handy -- that I find valuable in Perl/CAL development.

As I mentioned earlier, CAL is a standard COM interface, which means that it will work with any tool that supports COM. This includes some, but not all, versions of Perl. The version of Perl that comes with ClearCase, ccperl, does not support COM. If you want to use CAL with Perl, you will need to use another Perl distribution. I've found ActivePerl from ActiveState3 to be a great tool. It is a nice, robust implementation, and best of all it is free. Plus, the ActivePerl distribution includes a Perl package manager, which automatically finds and installs Perl packages. This is helpful for COM developers because you have to install a COM package in order to use COM in Perl.

The inexpensive Perl Development Kit, also available from ActiveState, is another extremely useful tool. If you plan to do Perl development in any serious sense, it is probably one of the best investments you can make. The two components I have used most are the Perl debugger -- which allows you to step through code, set breakpoints and watch variables -- and PerlApp, which allows you to convert a Perl application into a completely self-contained Windows program that can be run on any system, even if the system does not have a Perl interpreter.

The second edition of the Perl CD Bookshelf from O'Reilly includes searchable online copies of the following Perl references: Perl in a Nutshell, Programming Perl, 3rd Ed., Perl Cookbook, Advanced Perl Programming, and Perl for System Administration. This resource includes source code that you can cut-and-paste, which is an exceptionally common (and convenient) practice in Perl Programming. Legend has it that only one Perl program was ever written -- all the rest were derived from it. I take a copy of this CD everywhere I go, because it is just too useful to be without.

Some other tools and resources that are useful (but not required) in this context include:

● Rational SoDA. Rational SoDA is an automated report generator that has the ability to pull together static data from a variety of Rational tools. For example, it can reach into Rational ClearCase and Rational ClearQuest repositories, and build a report based on a template that you define. However, Rational SoDA cannot perform interactive queries and then take action based on the results -- something a Perl script can do

with ease.

● Rational ClearQuest API and Documentation. Like Rational ClearCase, Rational ClearQuest also offers a COM library that you can use to access its data. The Rational ClearQuest API is used by external programs to view or modify the data that Rational ClearQuest stores in the user database and schema repository. The sample Perl application we'll look at in this article, changeset_report.pl, uses the Rational ClearQuest API to query the Rational ClearQuest database.

● Visual Basic. I use Visual Basic to quickly develop prototypes and test programs. Visual Basic has extensive COM support and a built-in IDE (Integrated Development Environment), which makes it handy for troubleshooting any problems you might have with COM interfaces, including CAL. Visual Basic's object browser is also helpful for examining COM interfaces in an easy-to-understand format.

A Sample Project

Recently, while working with a customer, I came across a situation ideally suited to a CAL-based solution written in Perl. To understand this customer's particular needs, a little background will be helpful. Rational ClearCase and Rational ClearQuest store data in different data domains. Each tool provides extensive reporting capabilities within its respective data domain. Rational ClearCase uses cleartool subcommands such as cleartool find and cleartool describe for locating and reporting on ClearCase-controlled data, and more recent versions of ClearCase include ClearCase Report Builder, which uses a library of report programs to support dozens of standard report options. Rational ClearQuest uses Crystal Reports. There is, of course, integration between the two tools that allows them to share data. For example, if you enable UCM with Rational ClearQuest and Rational ClearCase, a link is made between change requests and activities, so that when you access an activity in Rational ClearCase, Rational ClearQuest gives you information about the change request or defect associated with that activity.

Rational ClearQuest maintains information about the process, the state of the activity, and so on; Rational ClearCase maintains the activity objects -- the actual change set information. In UCM, that information always stays in the Rational ClearCase database to avoid replicating information unnecessarily4. Now, let's say you were doing a code review, and you wanted a report that showed all the open and active defects in Rational ClearQuest, including the state of the defect, the owner, etc. -- and you also wanted to include the change set to show what code needed to be reviewed.

In fact, this is exactly what the customer wanted to do. The solution, as you have probably guessed, was to write a simple Perl script that uses CAL and the Rational ClearQuest API to create a report with just the change set information that they needed.

Figure 1: The Perl Application changeset_report.pl Uses CAL and the Rational ClearQuest API

The Perl application, changeset_report.pl, queries the Rational ClearQuest database via the Rational ClearQuest API to obtain a list of ClearQuest records. For each record, the program gets the Rational ClearCase UCM activity, and uses CAL to extract the change set information for that activity. The program then prints a report based on the data it finds.

Now let's look at the commented source code for changeset_report.pl. As we discuss the highlights and most important CAL-related aspects, you'll find it helpful to refer to the full code to better see how the pieces fit together. Because the application was actually developed for use in the real world, there are some added features that make it easier to use, but that you would not necessarily find in an example application focused solely on using CAL. For example, although all of the changeset_report.pl options and parameters can be set via the command line switches, there is also a Tk5-based graphical user interface (GUI) that prompts the user for any missing information (see Figure 2).

Figure 2: The Tk-Based Front End of changeset_report.pl

The sample program also includes an option for sending the output to an Excel spreadsheet instead of printing it out. Finally, although use of the ClearQuest API is an integral part of this sample application, we'll focus primarily on the sections related to CAL. In particular, the steps used to access Rational ClearQuest -- including creating a ClearQuest session, building the query, and getting results -- are not explained in detail here.

A Walk Through the Code

The first important line to note in the script is the first non-comment line:

use Win32::OLE;

Essentially, this line includes the Win32::OLE package so that it can be used in our script.

After the program has processed the command-line switches (or the input from the GUI), and builds a list of ClearQuest records, it enters a loop that is repeated for every ClearQuest record returned. In that loop, which appears considerably later in the script, CAL is instantiated for any ClearQuest record that contains a ucm_vob_object field. In UCM, the VOB object identifies the change set associated with a particular activity. If the ClearQuest results include an identifier for this object, then the script uses Win32::OLE to create a new ClearCase Application object, as you can see below.

#----------------------------------------------------------------# If the CC activity ID is not null, then we need to use CAL# to extract the changeset info from the CC activity object#----------------------------------------------------------------

print "Instantiating CAL\n" if $DEBUG;my ($CCApp) = Win32::OLE->new("ClearCase.Application") or die "Can't create ClearCase application object via call to Win32::OLE->new(): $!";

The two entry points for interacting with CAL are the Application object and the ClearTool object. Most applications -- including this sample application -- use the Application object, as it exposes most of the CAL API. The ClearTool object contains a single method, CmdExec, which can be used to execute a cleartool subcommand string. This ensures that any capabilities in cleartool not covered by the Application object are accessible programmatically. The code above instantiates a ClearCase Application object named $CCapp. The my operator simply limits the scope of the $CCApp variable. If the Application object cannot be created, then the script terminates or "dies" with an appropriate error message. The "print…$DEBUG" line in the code above and throughout the script serves as a rudimentary -- but very common -- debugging mechanism. If you don't want to use the Perl debugger, you can enable this verbose output by simply un-commenting the $DEBUG = 1 line located near the start of the script.

Once we have an Application object, we use the Activity()method to obtain an activity object from the value of the ucm_vob_object field that we obtained from

Rational ClearQuest, as shown below.

#----------------------------------------------------------------# Get an activity object from CAL #----------------------------------------------------------------

$myactivity = $CCApp->Activity($activity_id);

The result is an activity object, which I assign to $myactivity. To resolve the names of objects in an activity's change set, Rational ClearCase requires a view context. You can have ClearCase identify the best view heuristically, using the NameResolverView property of the activity. This view can, in turn, be used to resolve names of objects in the activity change set:

$view = $myactivity->NameResolverView;

To get the actual change set, we call the ChangeSet() method of the activity object, which returns a Collection object containing change set entries or CCVersions:

#----------------------------------------------------------------# Get the activity's change set, which is a CCVersions collection# Use the activity's "nameresolver view" for name resolution.#----------------------------------------------------------------

$ChangeSet = $myactivity->ChangeSet($view, "False"); $CS_Entries = $ChangeSet->Count;

At this point, the script enumerates the change set entries and accesses the path name of each changed source file via the ExtendedPath property.

#----------------------------------------------------------------# Loop through the CCVersions collection, collecting the names of # the versions for printing. #----------------------------------------------------------------

print "Getting ChangeSet info\n" if $DEBUG; $CS_Index = 1; while ($CS_Index <= $CS_Entries) { . . . $Version = $ChangeSet->Item($CS_Index); $VersionPN = $Version->ExtendedPath; . . . $CS_Index++; }

After iterating over each element of the change set collection, the script prints the results (or sends them to Microsoft Excel), and then repeats the process on the next ClearQuest record until there are no more. Sample output from changeset_report.pl is shown in Figures 3A and 3B.

Figure 3A: Sample Standard Output from changeset_report.pl

Figure 3B: Sample Microsoft Excel Output from changeset_report.pl

What's Next?

The changeset_report.pl script is not exceptionally complicated. In fact, it uses only a small subset of the capabilities offered by CAL. The CAL documentation includes a comprehensive list of all the CAL interfaces, as well as additional sample applications (including a couple more written in Perl). So, while changeset_report.pl illustrates only a fraction of what you can do with CAL, it does show how useful and practical CAL can be with a minimum of effort. With this small Perl program, we can see how easy it is to use CAL and the Rational ClearQuest API to access Rational ClearCase and ClearQuest data programmatically.

In fact, there are a myriad of different uses for CAL; and you've probably already thought of a few yourself. One possible use that I have started to play with is a kind of "release manager" application -- a utility that would present the user with a choice of baselines from which to construct a particular configuration and generate release notes based on the selected baseline. Although I have not finished it yet, working on it has reminded me how valuable CAL is, and how many possible applications it has. Once you start using it, I'm sure you'll find even more.

Notes

1 Larry Wall, Tom Christiansen, and Jon Orwant, Programming Perl, 3rd Edition. O'Reilly & Associates, 2000.

2 For example, Essential COM by Don Box (Addison-Wesley, 1998).

3 The author has no relationship with Active State. ActivePerl is available at www.activestate.com.

4 Note: In the non-UCM integration between Rational ClearCase and Rational ClearQuest, change set data is replicated into the ClearQuest database, so the sample application is not needed to generate such a report.

5 Tk is an exceptionally helpful package used to create graphical user interfaces. For more information on using Tk in Perl, see Learning Perl/Tk by Nancy Walsh, O'Reilly & Associates, 1999, or the Perl/Tk Pocket Reference by Stephen Lidie, also from O'Reilly & Associates (1998).

References

The Perl CD Bookshelf, 2nd Edition. O'Reilly & Associates, 2001.

Randal L. Schwartz and Tom Phoenix, Learning Perl, 3rd Edition. O'Reilly & Associates, 2001.

Dave Roth, Win32 Perl Programming: The Standard Extensions. Macmillan Technical Publishing, 1998.

Larry Wall, Tom Christiansen, and Jon Orwant, Programming Perl, 3rd Edition. O'Reilly & Associates, 2000.

Don Box, Essential COM. Addison-Wesley, 1998.

Nancy Walsh, Learning Perl/Tk. O'Reilly & Associates, 1999.

Stephen Lidie, Perl/Tk Pocket Reference. O'Reilly & Associates, 1998.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

A Physicist Looks at Project Progress

by Joe Marasco

Senior Vice PresidentRational Software

Projects tend to have a "percent completion" curve that is remarkably consistent from project to project. As a physicist, I have taken a look at this regularity and attempted to describe the underlying forces acting on the project by considering the derivatives of this curve. While extending the laws of physics to project dynamics may be somewhat speculative, I hope to help project managers understand the rhythm of their teams.

Have You Seen This One Before?

Most of the time, progress curves reported at the end of a project look like the one shown in Figure 1.

Figure 1: A Typical Project Percent Complete Curve

In fact, most projects are planned to have a progress curve that looks like this. Most wind up reporting progress that looks somewhat like the plan, with some distortion to reflect reality.

Because so many project completion curves look like this, I began to wonder if there are fundamental forces at work that cause the curve to be this way. How does a physicist approach this problem?

It All Goes Back to Newton

Some people claim that "modern" physics started with Newton, although the first modern physicist was probably Galileo. No matter. What Newton told us was that the acceleration of a body was proportional to the force applied to it, the proportionality constant being the inverse of its mass. The more force, the more acceleration.

The problem is that acceleration is something we experience rarely. We experience it when our car peels out at a stoplight, when we stomp down on the brakes, or when an elevator first takes off or comes to a sharp stop. More often, we are aware of two other variables, velocity (speed, in common parlance) and position (location, or where we are). We seem to be aware of going fast or going slowly, and we are usually aware of where we are. But force is only indirectly related to these metrics.

Derivatives, Rates of Change, and the Slope of the Tangent

For the calculus-aware, we know that velocity is the first derivative of position, and that acceleration is the first derivative of velocity, making acceleration the second derivative of position. If you never mastered calculus (or have forgotten it), substitute "rate of change" for "derivative." That is, your velocity is the rate of change of your position, and your acceleration is the rate of change of your velocity.

According to Newton, acceleration is proportional to the force applied, so the second derivative of the position of an object is proportional to the force. Another way of saying this is that you have to take the applied force and integrate twice to get information about the position. This is definitely not something that is intuitive for most of us. So we'll leave this integrating twice business alone for now, with the knowledge that there is sufficient mathematics to perform our integrations and differentiations once we understand the mechanisms.

For those of you who reason or think better graphically, remember that you can find the derivative or rate of change for any curve by looking at a line tangent to the curve at the point in which you are interested. For example, in the first example below -- the parabola -- the derivative or rate of change at the midpoint is zero. We can tell this because the tangent line to the curve there is horizontal, which has a slope -- or rate of change -- of zero. Prior to that, the tangent at any point has a positive slope; after the midpoint, the slope is negative.

A Simple Physical Example

A ball thrown vertically up in the air has a very well-defined trajectory.1 If we plot the height as a function of time, we get the nice parabola shown in Figure 2.

Figure 2: Trajectory for a Vertical Ball Toss

A note on the graphs in this article: we used Excel to plot them. We took the horizontal axis -- elapsed time -- to run from zero to a hundred percent, and then we used mathematical formulae2 to generate the data values. We always normalize the vertical axis to "one" -- that is, we have the height go from zero to its maximum, which we call "one." We computed all the derivatives numerically, once again using Excel to do the computations and the plots. Hence you can replicate all of this yourself and need no knowledge of calculus to derive the results.

Now if we plot the velocity of the ball3, we get the graph shown in Figure 3.

Figure 3: Velocity for a Vertical Ball Toss

Note that I have "normalized" the vertical axis so that the velocity falls between plus one and minus one; this helps us understand the concept by abstracting out the numbers4. This graph tells us that the ball starts out with a maximum velocity of one "unit," and that velocity decreases linearly until its reaches zero at the midpoint of its flight. This makes sense to us, because the midpoint of the flight is the highest point it achieves; at that instant, the ball is going neither up nor down, so its velocity is zero. Then it begins to fall, so its velocity is negative5. When the ball arrives at its starting point, its speed is exactly the same as when it was launched, except it is moving in the opposite direction.

Now, what about acceleration? Well, we just repeat the process one more time, obtaining the graph shown in Figure 4.

Figure 4: Acceleration for a Vertical Ball Toss

Once again, we have "normalized" the numerical value6 to -1. What this is telling us is that there is a constant acceleration. That is consistent with our model; close to the surface of the earth, the gravitational force is constant. And it is "negative" because we have chosen "up" to be positive, so the attractive force of gravity pulls the ball "down."

Note that this acceleration is negative for the entire flight of the ball -- even when the velocity is positive (upward flight), the acceleration is

negative. We are "subtracting" velocity throughout the flight: We start out with some positive velocity which reduces to zero at the highest point, and then we keep subtracting until the velocity is as negative at the end as it was positive at the beginning.

Whew! Newton is vindicated. We have "experimentally" verified that a parabolic position graph implies a constant acceleration, which in turn implies a constant force.

Can we use reasoning of this type to understand the "forces" at work on a project? As with acceleration, we do not experience these forces directly. Rather, we observe things that are indirectly related. So let's return to the percent complete curve; it describes what happened, plotting the combined results of all those forces we don't directly observe. If we assume this project completion graph is prototypical, then what would that tell us about project dynamics?

The "Project Completion" Graph: Plotting Position Against Time

As we have already pointed out, most projects can be plotted according to the following graph:7

Note that we have once again normalized things, so that the time axis goes from zero to a hundred percent, and the vertical axis also goes from zero to one hundred percent.

Also note that we have somewhat arbitrarily positioned 50% complete at 50% of the elapsed time. The symmetry of the curve is a simplifying assumption, but one that is useful for this exposition. For the time being, we have no reason to assume otherwise.8

The Curve of Human Behavior

Generically, these curves are referred to as "S-Curves"9 because they resemble the letter S if you stand back a bit and squint. They describe a

wide variety of human behavior; the classic "learning curve," for instance, assumes this shape. The notion that a project completion curve mirrors a learning curve ought to tell us something.

When we begin to learn something new, we spend a lot of time fumbling around without understanding much. This corresponds to the first, "flat" part of the curve, where time goes by without much progress. Then, as we begin to understand what is going on, our progress increases; this is reflected by the second part of the curve, sometimes called "the ramp." During this period we actually make a lot of progress per unit time, and if we could continue at this rate we would finish our task sooner. But inevitably, we enter the third part of the curve, which goes flat on us again. What is happening here is that we "plateau"; we stop making dramatic progress, and the last few percent (learning all those remaining annoying details) takes a long time. This learning experience is repeatable over and over again, from domain to domain.

When the ramp is gently inclined, we talk about an "easy" learning curve; when the ramp goes up very fast, we talk about a "steep" learning curve. Many people cannot handle steep learning curves; they need more time to assimilate the bulk of the new knowledge.

"Quick studies" are interesting people; they have a very short approach period, go up a very steep ramp, and then quit without ever learning the last few percent. By truncating both flat parts and navigating steep ramps, they can radically abbreviate the time it takes them to acquire knowledge and skills. But they are rare indeed. Most of us do best with two flat parts and a moderate ramp.

Projects follow a similar rhythm. Early in the project, you make progress slowly. There is a lot of planning, organizing, and discovery that causes you to expend a lot of effort without making much tangible progress. Once you get going, you get on the ramp, and progress feels (and probably is) somewhat linear (although we know it is not quite linear). Late in the project, things slow down again, which explains why finishing is hard. Once again, nailing down all the details you need to complete to deliver the product takes a relatively long time to do. That is why a project that is 90 percent done is probably low risk from a technical point of view, but will still take a while to finish. Things slow down at the end, and there is almost nothing you can do to fix that. Adding people at this stage, for example, will just push the end date further out, as has been shown over and over again.

What about new product acceptance? Same thing. Slow at first, as people have to find out about the product, understand its costs and benefits, talk to their colleagues, and go through a selection and acquisition process. These are the early adopters. Then there is a ramp, when the product "catches on" and demand seems to go through the roof. This is the point Geoffrey Moore calls "crossing the chasm,"10 when a product goes mainstream. This goes on for a while, and then sales begin to slow. We have now hit the third part of the curve, the plateau. The product is now deemed "mature," with trailing or late adopters being the principal buyers. Very likely, another rival product is hitting its ramp at this point and taking away market share from the earlier product. So this universal curve also

represents a product's sales lifecycle.

The Project Velocity Curve

Before getting into myriad debates about how "real" this curve is, let's just assume that it represents the majority of reported "percent complete" data and see what can be inferred. Like many other examples in physics, we first try to understand the idealized behavior, and then modify our conclusions by adding back in the effects of air resistance and other real world phenomena to the model.

The crucial observation is that this curve is a "position versus time" plot. In other words, this curve plots our "position" at any point in time. Now, what does the "velocity" curve look like?

Mimicking the steps used above, we have the technology to answer that question.

Taking the rate of change of our project completion curve yields the graph shown in Figure 5.

Figure 5: Velocity for Project Percent Complete Curve

By now you are used to having the vertical axis normalized. What this graph tells us is that projects start off slowly and move faster for a while. Then they reach maximum velocity (in Figure 5, maximum velocity is achieved at the halfway point, but that is because our project completion curve is symmetric), and then start to slow down. As you cross the finish line, you are actually going fairly slowly, much as you did at the very beginning.

This is consistent with our common, everyday experience. Projects are slow to get started, they seem to "gather momentum" at some point, and then they "bog down," and things go more slowly. Finishing is hard. Getting all the fine details worked out so that you can deliver the product seems to take forever. Often it feels like you stagger across the finish line.

Once again, the exact values for this curve may vary, but the pattern,

from project to project, is remarkably consistent. So what does this say about the underlying "forces" driving the project?

Well, at this point the physicist would take another derivative. Let's do that!

The Project Forces Graph

The resulting acceleration curve, which implies the forces at work, looks like the one in Figure 6.11

Figure 6: Acceleration for Project Percent Complete Curve

What are we to make of this?

Well, here's one interpretation: During the first third of the project, we see an increasing positive force. This corresponds to a lot of enthusiasm for the new project, the addition of new team members, and a general optimism about the chances of success. This might cynically be called the "ignorance is bliss period." It is this positive, increasing force that causes the project to gain velocity (sometimes colloquially called "momentum," or "the big mo").12

At about the one-third point, this positive force begins to decline. This could correspond to reality setting in; people are far enough along to start to understand what the real problems are and are beginning to feel some schedule pressure. After all, they have now used up one third of the time but still have mountains of work left to do. Also, the team is beginning to feel the full burden of a large staff; a lot of time is spent in meetings, and communicating information to all the members of the team grows difficult.

And then we reach the halfway point. At this point, according to the graph above, a negative force sets in. The team begins to feel the project "pushing back." Many of the really tough problems are not succumbing to solutions as quickly as we thought. People begin to really panic as more and more sand slips through the narrows of the hourglass. At about the two-thirds point, the force hits its maximum negative value; there is this sensation of swimming in molasses. If the project stays here for very long,

it dies.

And then, when we most need it, the project "turns the corner." There is a breakthrough, and all of a sudden, things don't look quite so bleak anymore.13 While there is still a negative force at work -- the knowledge that there are still a million details left to complete, and not much time to do it in -- this negative force decreases. The main reason for this is that the team can see the finish line. The negativity decreases until we get across the finish line.

Be aware that the actual "turning points" on this curve -- one third, one half, two thirds -- will vary from project to project, and will in turn affect the velocity curve and the project percent completion curve. There is nothing magic about the one-third, one-half, and two-thirds completion points; these result from the symmetric project completion curve we originally chose for ease of exposition. The exact points on our project are unknowable; all we can "predict" is that it will go through these phases.

This is kind of gratifying. We start with a prototypical percent complete curve, take a couple of derivatives, and infer the forces underlying the project. The data seems to empirically fit the theory. Have we really accomplished anything here?

Reality Intrudes

A physicist looks at a theory from a number of different points of view. It is always dangerous to extrapolate physical ideas like position, velocity, and acceleration to group dynamic variables like "project percent complete," "project velocity," and "project forces." We must be extremely careful not to attribute more "science" to this than can really be there.

On the other hand, the analogy seems to give results that are consistent with reality. So let's move to the next step.

A physicist judges a theory by its ability to explain and predict. That is, a theory must first explain the results of all known experiments. If experimental results exist that contradict a theory, then the theory is wrong, unless you can go back and show that the experimenter made a mistake. In our example, we have seen that most projects have a similar percent complete curve, so all that will change from project to project is the details of the transition points of the derived curves.

And that brings us to the limited utility of the theory. What we would like to be able to do is to predict the percent complete curve while we are at some state of incompletion. Or to put it another way, we want to know when we are going to be finished. At any point in time, all we have is that portion of the curve "behind us," and some notion of the velocity curve. The "forces" curve is pretty hard to pin down quantitatively. In fact, it is often hard to understand the velocity curve, although metrics that could indicate velocity would be very, very useful. Knowing your "position" is important, but to more accurately forecast, you need both actual position and actual velocity.

Today, many of our project metrics focus exclusively on position -- "Where are we?" As we can see from this discussion, having velocity metrics -- rate of change of position, how fast are we going? -- is equally important. And should we ever get to the point of being able to understand how the velocity is changing, then we would have an even more complete picture. So we need to think about project metrics collection in this light.

What About Iterative Development?

All that we have said is somewhat introductory to the notion that we don't do projects in one fell swoop, but break them down into iterations. Each iteration has its own rhythm, so the graphs described above are really only an approximation. For example, for a project with four iterations, the percent complete curve would probably look more like Figure 7.

Figure 7: Percent Complete Curve for a Four-Iteration Project

Here what we see are four S-curves stacked one on top of another. Percent complete is cumulative, and we have made the assumption that each iteration takes 25% of the time and gets us 25% along on the completion curve. In the following sections we will depart from this simplifying assumption.

Now let's jump to the velocity and acceleration (forces) curves that correspond to this four-iteration percent complete graph.

Figure 8: Velocity and Forces in a Four-Iteration Project

What do these graphs of the derivatives tell us?

What we see is that the velocity curve replicates itself four times -- no surprise there. The project gains and loses velocity four different times, corresponding to each iteration having a somewhat similar rhythm. The second derivative curve -- acceleration14 -- has three discontinuities, which we can also "see" in the velocity curve.15 What this means is that at the start of each iteration after the first, you need to "kick" the project out of its residual negative force from the previous iteration and impart an initial positive force. This is what causes the velocity vector to change abruptly. Without this instantaneous, discontinuous force, you won't get going again without losing time.

Most good project managers know about this and apply the requisite force at the appropriate time. We shouldn't be all that surprised by the discontinuity -- after all, we started at zero with a non-zero positive force to get the project underway. All we are saying is that at the end of the iteration we still have a residual negative force, and we need to "jump start" back to that positive force we began with.

Having applied this positive force, you need to once again build it until the iteration hits its "wall," and the inevitable negative forces set in. Then it's just a question of hitting the breakthrough for that iteration, the one that again reverses the negative force curve.

So now we know how to apply our model to a multiple iteration project. It's the same game: Plot the percent complete curve, take the derivatives, and interpret the results. The only thing that has changed is that our percent complete curve is a little more complex at the outset. But the rules of the game are the same.

But all this is a bit boring when all the iterations look the same; not to mention the fact that we have no insight regarding the real project at hand, because thus far in our analysis nothing truly mirrors reality. So let's get real.

Iterations and Phases

In real life iterative development, we may have many iterations, divided up into distinct phases. How can we inject this sort of reality into our model?

Well, I'm not up to a multi-phase, multiple-iterations-per-phase graph. And it's not clear that we need that complexity anyway. After all, it is the phases that have the biggest difference in their characteristics, not the iterations within the phases. So, to keep things reasonably simple, let's model the phases and assume only one iteration per phase.

The Rational Unified Process® defines four phases:16

● Inception

● Elaboration

● Construction

● Transition

These four phases do have different characteristics. Early in the project, we are doing a lot of discovery (new learning). In the middle of the project, we do somewhat less discovery but lots of invention; there is still a lot of learning going on. Later in the project, we are actually doing the activities that count for "completion"; some invention and lots of implementation. Learning drops off a bit as the project moves toward completion.17

At this point it is necessary to decouple the ideas of "learning" and "completion." Up until now we have been assuming they are the same. Here are some numbers that my colleague Philippe Kruchten18 believes are applicable to the four phases:

Table 1: Learning and Completion Metrics for Phases in the Rational Unified Process

Phase Percent of Elapsed Time Percent Learned Percent Complete

Inception 0 - 10 0 - 10 0 - 5

Elaboration 10 - 40 10 - 60 5 - 25

Construction 40 - 90 60 - 90 25 - 90

Transition 90 - 100 90 - 100 90 - 100

What these numbers tell us is that we learn faster than we achieve completion when we do phased, iterative development. This accelerated learning is what helps us reduce risk. How do we introduce these ideas into our model?

We have only two modifications to make:

1. We are going to plot "percent learned" and "percent complete" separately.

2. For each of these curves, we will plot the S-Curve segments according to Table 1, in order to model that level of reality.

Results

Here they are. We'll present all the results together in Figure 9 and then open up the discussion. We include Philippe's table again so that it is easy to remember the phases.

Phase Percent of elapsed time Percent Learned Percent Complete

Inception 0 - 10 0 - 10 0 - 5

Elaboration 10 - 40 10 - 60 5 - 25

Construction 40 - 90 60 - 90 25 - 90

Transition 90 - 100 90 - 100 90 - 100

Figure 9: Graphs for Table 1 -- Learning and Completion Curves

Discussion of Results

There would appear to be a lot of explaining to do.

First, let's address the difference in the shapes of the learning and completion curves. Basically, you achieve 60% of your learning in the first 40% of the project, and are only 25% complete at that point. This reflects the notion that in iterative development we emphasize learning early to reduce risk. The counterweight is that you don't make much "visible" or "tangible" progress, because often the learning does not have many artifacts that go along with it. In any event, as a project manager you are

going to have the following conversation at the 40% point of your project:

Boss: "You've used up 40% of the time, but you show only 25% complete. You are in big trouble."

You: "Not really. In iterative development, learning is important in the first half. And we have accomplished 60% of the learning we set out to do."

Boss: "Really? That's great. Can you show me the 60%?"

At this point I would suggest having some additional arrows in your quiver.

Now let's look at the velocity curves. Let's defer discussion of the first and fourth phases to later, because it's more interesting to discuss these and their associated forces at the same time. For the moment, let's concentrate on the Elaboration and Construction phases, where there are interesting differences.

What we see is that learning velocity peaks during Elaboration, whereas completion velocity peaks during Construction. This is consistent with our model, in which we set out to get over the learning hump in Elaboration, sacrificing or deferring completion a bit. The idea is that the measurable artifacts of completion only begin to really show up during Construction, so that is when that curve shows its peak.

We might ask why both velocity curves go down to almost zero at the 40% point and then back up. Wouldn't it be better if we could "keep" some of that velocity across the boundary? Yes, it would. The reason we can't is that the two S-Curves join at the boundary. In fact, discontinuities in the force curve show up at the boundaries for this very same reason. The reality that corresponds to this mathematical artifact is that distinct phases cause disruption on any project. That is why many senior project managers try to blend things at the boundary; they sometimes try to get an "advance team" working on the next phase a little early (before the antecedent phase is actually done) to smooth the transition over the boundary. It is hard to pull this off. The other logical consequence of this effect is that the more phases you have, and the more iterations you have within phases, the more boundaries you create. If there is a fixed overhead at each boundary, then you increase your total overhead by increasing the number of phases and iterations.

The forces comparison for Elaboration and Construction is also pretty clear; the learning forces are stronger during elaboration and relatively weak during construction. On the other hand, we note that the completion forces have about the same peak values during both Elaboration and Construction, which is a good thing.

Now let's address the Inception and Transition phases. Each of these is accomplished over a short time interval, only 10% of the project each. Because the S-Curve causes its derivative curve to have a peak, accomplishing that turnaround in a short period of time causes the peak to be bigger. Correspondingly, the force curve necessary to turn around the

velocity curve in this short interval also requires large peak forces. This is why when we get to the force curve, both for learning and construction, we see maximum peak forces in these intervals. It is a result of having a very short interval. The antidote, if you will, is to not have S-Curve behavior during these short periods. If the progress curve here were simply linear over the interval, then the force would be less. S-Curves and their derivatives do better over longer time periods. They are perhaps the right model for long periods, and a poor model for short intervals.

On the other hand, these "large" forces may be real and not artifacts of the S-Curve compressed into short intervals. It is certainly the case that we have large forces at the end of the project, during Transition. Anyone who has ever had to finish a project will attest to that; it's what makes finishing so hard -- you need a big push when the team is the most tired. And, there are also large forces early in the project, during Inception, because it is at the end of Inception that we have our first "go/no go" decision to face. If people working on the project are concerned about cancellation at this early branch point, then you can be sure that there will be large forces at work.

One Last Graph

There is one other graph that is perhaps of interest. Let's assume that we can "add" the completion forces and the learning forces together. Force is a vector, so the assumption we are making here is that the completion forces and the learning forces are co-linear, or acting in the same direction, a somewhat arbitrary assumption. However, if we make this assumption, then we can sum the forces and get the total force on the project as a function of elapsed time, as Figure 10 shows.

Figure 10: Total Forces on a Project (Learning + Completion)

Let's look at some areas under the curves in the four regions.19 We note that all the segments have the same form, so let's look at the peak value of the total force and the length of time the segment is in play; the product of these two will be proportional to the area, as Table 2 shows.

Table 2: Product of Total Force Peak Value and Time Interval Length

PhasePercent of

Elapsed Time

Length of Time

Interval

Peak Value of

Total Force

Product (Impulse)

Inception 0 -10 10 0.8 8

Elaboration 10 - 40 30 0.4 12

Construction 40 - 90 50 0.2 10

Transition 90 - 100 10 1.0 10

For the mathematically and physically inclined, we know that the area is an integral, and the integral of force over time is equal to the change in momentum.20 What we see from the table is that the change in momentum is roughly the same over the four phases. The change in momentum (or impulse) is the same during Construction and Transition -- ten units. The biggest impulse is during Elaboration, twelve units. And the combined impulse of the first and second phases -- twenty units -- is equal to the combined impulse of the third and fourth phases. Half the total impulse is applied during the first 40% of the elapsed time; this again expresses the idea of "front loading," which we believe is a good thing.

Note also that the discontinuity in the total force curve at the 40% boundary is not too large. That is probably also a good indicator.

So the conclusions drawn from using Kruchten's and Royce's numbers for learning and completion percentages in the four phases are consistent with the benefits we believe we see in iterative development projects. While we have made many assumptions and pushed the model about as far as we are comfortable with, the results do appear to be both internally consistent and consistent with observed behavior.

Summary

We have taken a simple, universal model -- the S-Curve -- and applied it to software project management. We used simple physics to infer from the S-Curve the velocity and acceleration, and hence the forces, at work on the project. We extended the model to multiphase iterative development and found results that are consistent with observation. We even attempted to separate learning from completion metrics and separately analyze those forces. Finally, we added the forces together and drew some conclusions about the total force as a function of time, and the impulse applied to the project during its various phases.

Much of this analysis is speculative, and we encourage you to ponder this and give us alternate interpretations of the graphs we have drawn. I hope that many of you will find other nuggets in here that neither my reviewers nor I have found yet.

In the meantime, the conceptual model can make us ready to anticipate

and deal with the various phases of the project as they play out. Knowing that there is some degree of "determinism" at work can sometimes be a comforting thought when a project hits its low point and you're worried that the team may lose hope.

Notes

1 For the purposes of this example, we will ignore air resistance. As in all physics experiments that make this assumption, this has the disastrous side effect of killing the experimenter, as he has no air to breathe.2 If you really, really, want to know the formulae I used, send me an e-mail.3 We do this by numerically differentiating the data points from the parabolic curve. That is, we take successive differences of the values of the first curve, then multiply by a constant to get the values to fall between plus and minus one. The careful observer will notice that this causes us to "miss" the first point on the first derivative curve, and the first two points on the second derivative curve, and also cause a very slight shift (half a bin, actually) on the derivative curves. This won't become noticeable until later; it doesn't affect the arguments at all, but is just an artifact of how we are computing the derivatives.4 One convention is important: the "up" direction is positive. This will determine which velocities are positive and negative.5 Hence the difference between velocity and speed. Speed is always positive; as the ball falls, it speeds up. However, its velocity becomes more negative. The number representing speed and velocity gets larger in magnitude; however, velocity takes into account direction, so it becomes a larger negative number.6 Our friend Excel insists on calling zero "-0." We are smart enough to know that there is only one zero, and it doesn't have a sign.7 Here come the caveats. This is of course an idealization. Real life curves are rarely so smooth. Also, projects don't uniformly move forward, although it is rare to see a reported percent completion curve show a dip. But the real problem is that there is no single metric we can use across the entire project to capture "percent complete." One of the largest problems on any project is figuring out how to differentiate reported progress from actual progress.8 You may see curves that look like this that have various other "percent completes" at 50% of the elapsed time. It is allowed. We don't deal with them here for simplicity.9 On Sesame Street, we would say that this section is brought to you by the letter "S".10 Geoffrey A. Moore and Regis McKenna, Crossing the Chasm : Marketing and Selling High-Tech Products to Mainstream Customers, Revised Edition. Harperbusiness, 1999.11 The slight shift mentioned in footnote 3 is noticeable here; the forces curve should cross the axis at 50%. Once again, this is an artifact of the crude way in which we compute the derivative, and is absolutely irrelevant to the argument.12 Physics seems to intrude into our language even when we don't want it to.13 The more crusty project manager may be heard to murmur at this juncture, "We may finally have broken the back of this son of a bitch."14 By now you are used to the idea that the graphs can be labeled "acceleration" or "force" interchangeably.15 It may be hard to see, but there is a difference between the four peaks at 100% on the velocity curve and the three troughs at about 10%. The passage through the maxima is smooth, with a zero derivative. On the other hand, the passage through the minima is not smooth; the slope changes abruptly. That is what leads to the discontinuity on the acceleration (force) curve.16 Philippe Kruchten, The Rational Unified Process: An Introduction, Second Edition. Addison-Wesley, 2000.17 The relevant reference here is: Grady Booch, Object Solutions: Managing the Object-Oriented Project, p. 61. Addison-Wesley, 1995.18 The "completion" numbers come from work by Philippe Kruchten and Walker Royce in 1999 on the Rational Unified Process. See footnote 16. The "learning" numbers are from a private communication with Philippe, and represent his best estimate.19 Here we will consider all areas to be positive. That is, we take the absolute value of areas below the axis, so that there are no "negative" areas. If we didn't do this, the total area would integrate to zero.20 More precisely, the integral of the force over time is the impulse, which is equal to the change in momentum.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information

Reader Mail

Dear Rational Edge:

I am involved in a research project for the University of Castilla la Mancha in Ciudad Real (Spain). We need to extend Rational Rose functionality with an add-in. We studied Jim Conallen's example at http://www.therationaledge.com/rosearchitect/mag/archives/9904/f5.html, and we decided to follow this technique. But we have the problem of not knowing the classes that we have to manage, so we would like to know where to find a complete REI class diagram.

If you could help us we will be very pleased.

Yours sincerely, Carlos Medina GallegoTecnólogo por la UCLM.

Jim Conallen answers:

Hi Carlos,

Actually, what I like to do is make a Windows shortcut to the file: ..\Rose\help\scrptref.hlp

It's the Rose REI help file, and it is well-documented. While this help file is a little light on UML class diagrams, it does contain a lot of examples. In fact, I learned how to code Rose add-ins from this one file exclusively (I never took a course on the REI, and had to figure it all out myself: [pat on back] ).

All documentation for the classes you can manipulate are under the Rational Rose Extensibility Reference section. I use this file so often that I made a short cut to it from my main Windows tool bar. I made a shortcut to this one help file shown below (install of the parent help file, to make it easier to do searches in only the REI section).

Hope this helps,

Jim

A reader from Houston, Texas writes:

Many thanks to Joe Marasco for the article on politics in a technical organization. I recently left Rational to "do missionary work" at the Harris County Justice System in Houston, Texas. My role here is enablement of the Rational tools and methodologies. I distributed the article to my boss and other major stakeholders and they loved it.

Thanks again.

For more information on the products or services discussed in this article, please click here and follow the instructions provided. Thank you!

Copyright Rational Software 2001 | Privacy/Legal Information