Process Principles for COTS-Based Systems

© 2003 by Carnegie Mellon University page 1

Process Principles for COTS-Based Systems

5 September 2003

Software Engineering InstituteCarnegie Mellon UniversityPittsburgh PA 15213

Sponsored by the U.S. Department of Defense

Tricia Oberndorf


Agenda

• COTS Background

• A/APCS Framework

• CMMI Background

• Implications of Using CMMI for COTS-Based Systems


COTS Background

Building systems today:• Few entirely custom-built to order• Commercial products expected to play major role

Diverse things influence the shape and function of a COTS-based system (CBS):• Stakeholder needs• Product characteristics and marketplace dynamics• Degree of interaction with legacy systems

Together, these compel many changes in system development and maintenance methods for CBSs.


What is a COTS-based system?A COTS product is one that is …• Sold, leased, or licensed to the general public• Offered by a vendor trying to profit from it• Is supported and evolved by the vendor,

who retains intellectual property rights• Available in multiple, identical copies• Used without modification of its internals

A COTS-based system is one that … • Uses one or more off-the-shelf

components from one or more suppliers plus any needed custom components

• Is integrated to achieve new or expanded system functionality

Lega

cy

Reuse

ass

ets

ComponentsComponents

COTS-Based System

COTS-Based System

??????

COTS

pro

duct

sCus

tom


Conceptual Bases: Requirements1

“Nail down the requirements first” will not work – the marketplace will not cooperate• Think of it as a new sphere of influence:

Constraints come from stakeholder needsAND marketplace imperatives

System that is created mustaccommodate competing sources of gravity


Conceptual Bases: Requirements2

Requirements (cont.)• Must distinguish between negotiable and non-

negotiable• Keep the non-negotiable set as small as possible• Understand how to prioritize and trade-off the

negotiables

A risk-driven spiral approach is key:• Frequent use of prototypes• Considerable interaction with system’s end users• Gradual refinement of understanding of system


Conceptual Bases: Knowledge

CBS development and maintenance is dependent on an evolving Body of Knowledge.

Architecture anddesign constraints

End-userbusiness processes

Acquisitionconstraints

BoK

Prioritized negotiables Marketplace offerings

Evolving System View

Non-negotiables

Legacy system context

Risks


Marketplace Affects CBS Approach

Traditional Engineering Approach

Requirements

Architecture & Design

Implementation

Requirements-driven Negotiation-driven

Required CBS Approach

Simultaneous Definition

and TradeoffsMarketplace

Stakeholder Needs/Business Processes

Architecture Design

Programmatics/Risk


The Process Framework

Based on these realizations and concepts, we have defined a framework that sets out the basics of a process that would be appropriate for the creation of a COTS-based system:

A/APCS: The Assembly/Acquisition Process for

COTS-based Systems


Process Overview

Three classes of activities:• Iterative: short-term, engineering-oriented

- Discovery: gather and refine system knowledge- Assembly: construct a prototype- Assessment: determine success of iteration & plan

• Pervasive: long-term, organizational scope- e.g., CM, license management, vendor relationship

management, contract tracking and oversight• Executive: event-driven, deal with decision making

- e.g., cost estimating, contract negotiation, project oversight

Today we focus on the Iterative.


Discovery

Gather and Refine activities occur simultaneously.

Gather:• Sources take many forms.

- Stakeholders, business processes, legacy systems, COTS marketplace

• Each is independent of the others.• There is no optimal order.

Refine:• Analysis and harmonization of the gathered knowledge• Yields the technical definition of the emerging system• Finds gaps, negotiates conflicts


Discovery Activities

Data from: stakeholders products …

RefineGather

Negotiate conflicts

Gap - need more data

Continue Discovery

Knowledge is sufficient for constructing executable

Go on?

Harmonized data

Mismatch

Agreement

Gather data zGather data y

Gather data xAnalyze


Assembly

Produces a prototype that reflects what’s been discovered

Reflects a traditional sequential process• Guided by local “ candidate requirements”

- Iteration Objectives, Detailed Iteration Plan plus hypotheses and desired behaviors expect to derive from prototype

- Vets “candidate requirements”• Produces an executable version of full system

- Likely to have less than complete functionality- Executable, not paper or specification or small piece

Does not preclude prototyping “in the small” throughout


Evaluation and Assessment

Occur at two levels:• Evaluation of the prototype in its own context

- Measures actual outcome against expected outcome- Considers degree to which prototype satisfies its

local “requirements”• Assessment of the iteration itself

- Addresses issues at project (not iteration) level- Considers whether objectives were good ones- Determines whether iteration results indicate

changes in project schedule, budget, etc.


On-going Iterations of A/APCS

Prototype

Deployed System

Body of Knowledge

Body ofKnowledge

BoK


Key Aspects of COTS Approach

• Simultaneous Definition and Trades• Parallel Business Process Engineering• Negotiable Requirements• Flexible Architecture• Current Market Knowledge• Integral Programmatics and Risks• Continuous Stakeholder Involvement• Disciplined Spiral or Iterative Practices• Frequent Executables


CMMIGuidance for improving an organization’s processes and ability to develop, acquire, and maintain products or services

Provides guidance for developing processes • is not processes or process descriptions • will not map one to one with the processes in your organization

Integrates four disciplines (bodies of knowledge)• System Engineering• Software Engineering• Integrated Product and Process Development• Supplier Sourcing

Expects all practices to be interpreted using knowledge of• CMMI• the organization • the business environment

CMMI process areas must be interpreted for each targeted domain


Presentation Format

Work Process Implications• High-level guidance for selected CMMI Process Areas or

disciplines- Highlight particular relevance- Provide unique interpretation- Suggest new process areas- Correct misconceptions

• Guidance for additional CMMI Process Areas for later reference

• Important process considerations for a COTS-based system approach

Take-away message for each aspect




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Simultaneous Definition and Trades

Selected Work Process Implications• Decision Analysis and Resolution – continuous stakeholder negotiations require

robust, agreed-upon decision processes• Technical Solution – alternative solutions (including COTS product selection) are

analyzed continuously to accommodate new information• Risk Management – a key project risk is over-defining one sphere without adequate understanding of

implications on other spheres • Project Planning – project activities start concurrently with extensive interaction among them from

project start until the system is retired • Verification and Validation – continuous determination that information in each sphere is sufficient,

complete, and meets operational needs

• Balanced consideration of four diverse spheres of influence- Simultaneous discovery of information in each- Decisions in one inform and constrain decisions in others- Continuous identification and analysis of trades- Trades negotiated as definition of the solution evolves

Continuously reconcile what stakeholders want with what is available




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Parallel Business Process Engineering

Selected Work Process Implications

• CMMI does not address changes to end-user business processes; expand concepts in Organizational Process Focus to plan and implement end-user business process improvement

• Organizational Environment for Integration – a shared vision of success among stakeholders, with suitable incentives and leadership, is critical to aligning business processes with alternative solutions

• Project Planning/Integrated Project Management for IPPD – implementing agreed upon business processes must be integrated in system planning

• End-users must be willing/able to change business processes to match those in COTS products

• Business process changes must be explicitly managed and coordinated as part of the project

• Business processes may continue to change with new COTS releases or new COTS products

Align business process engineering with system engineering




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Negotiable Requirements


• Requirements Development – must prioritize requirements to aid tradeoffs- stakeholders agree on minimum set of critical “must-have” requirements - evolvability is a high-priority, required quality attribute

• Requirements Management – disciplined and controlled management of requirements begins at project start to track identified and negotiated trades

• Project Planning/Integrated Project Management for IPPD – managing the project to encourage and reinforce the continual discovery of requirements while establishing sufficient stability to deliver a solution is challenging

• Requirements must be flexible enough to leverage available and projected COTS products

• Committing to a requirement is premature until COTS product behavior is understood

• As marketplace continues to change, requirements must be renegotiated

Requirements formation is a journey of discovery




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Flexible Architecture


• Technical Solution – each COTS product release requires evaluation and analysis of alternatives to support any incorporation decision-Describe behavior and linkage among system components for each

• Product Integration – composing and evaluating executable representations from project start is critical to verify and validate architecture suitability and evolvablity

• Project Planning/Integrated Project Management for IPPD – appropriate skills and resources are necessary to form, evaluate, and maintain system flexibility

- include effort to create and maintain “wrappers” or “glue” and re-integrate solution as COTS products change

• Architecture is considered early; evolves and is maintained until the system retired

• Potential drivers of change accommodated in architecture

Architecture is created and maintained as a corporate asset




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Current Market Knowledge


• Supplier Agreement Management – license agreements with COTS vendors must be negotiated to meet project needs

• Integrated Supplier Management – partnering with key vendors is critical (not just supplier management)- vendors seldom allow process monitoring; use hands-on evaluation- relationships with vendors’ other customers helps to amplify leverage

• Technical Solution – modification of COTS products introduces long term maintenance considerations and sizeable risk to project; avoid if possible

• Project Planning – significant resources may be required to monitor the marketplace and conduct COTS product evaluation; including experimentation facilities

• Anticipate and track changes to relevant market segments until system retired

• Anticipate and prototype system changes from updates to COTS products critical to the system

• Influence (not direct) COTS product changes, technology investments, and standards development

Marketplace is proactively monitored




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Integral Programmatics and Risks


• Technical Solution – engineering trades should include risk, cost, schedule and other programmatic factors associated with each alternative solution

• Project Planning – estimates of work product and task attributes should be generated for each alternative

• Analysis of alternative solutions includes team skills and expertise, cost, schedule, and associated risks for - building, fielding, and supporting the system - implementing any needed changes to business processes

(functional, operational, support)

Programmatic factors shape technical alternatives




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Continuous Stakeholder Involvement


• IPPD discipline – integrated teaming among disparate stakeholders throughout the development and maintenance is essential

• Validation – end users must always be involved in validating solution suitability

• Integrated Project Management for IPPD – accommodates resources for stakeholder involvement

- necessary changes to end-user operational processes must be explicitly and continuously managed and coordinated with solution development

Required stakeholder commitment may be unprecedented

• Significant commitment from all stakeholders required- identify, evaluate, and select alternative solutions - confirm results of any and all negotiations - agree evolving system meets their needs

• Stakeholders must reflect full diversity of interests




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Disciplined Spiral or Iterative Practices• Continuously determine a compatible and feasible

set of: business processes, requirements, plans, design, COTS products, and other components

- Enterprise business objectives drive solution definition- Risk considerations drive degree of detail - Marketplace dynamics drive development and maintenance

processes

Spiral development facilitates developing a viable solution


• Project Planning – if not already implemented, extensive effort may be needed to revamp planning and engineering processes for a spiral development approach

• Risk Management – tracking effectiveness of risk mitigation is key- highest priority remaining risks should be used to (re) direct and manage

the project




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Frequent Executables• Frequent executable representations reduce risk

and reduce misunderstandings- Provide critical insight into the solution’s behavior- Explore critical system attributes - Validate end user business process - Verify technical viability

Executable representations demonstrate stakeholder buy-in


• Requirements Development, Technical Solution, and Product Integration – each iteration should produce an executable representation reflecting current understanding of requirements, COTS products, business processes, and alternative designs explored and negotiated




Architecture Design

Programmatics/Risk




Architecture Design

Programmatics/Risk


Conclusion

CBS:• changes many things about your approach• requires even more discipline than custom

development

Although it needs interpretation, CMMI is a sound basis for CBS process improvement. • Applying CMMI to CBSs is more than just Supplier

Management:- All four disciplines- All process area categories

• Process areas will need to be integrated and applied differently

• Must add in your own activities to plan, design, and deploy changes to enterprise processes


For more information:

TR coming: CMU/SEI-2003-TR-022

www.sei.cmu.edu/cbs

Tricia Oberndorf Lisa BrownswordDirector, Dynamic Systems Sr. MTS, CBS (ISIS)412-268-6138 [email protected] [email protected]

Process Principles for COTS-Based Systems

Documents

Transcript of Process Principles for COTS-Based Systems