Post on 19-Jan-2018
description
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 1
World-Class Quality
Measurably Improving Your Requirements Based on the CMMISM
Tim Olson, PresidentQuality Improvement Consultants, Inc. (QIC)(760) 804-1405Tim.Olson@qic-inc.comwww.qic-inc.com
2003 CMMISM Technology Conference
® CMM is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.SM CMMI is a service mark of Carnegie Mellon University.QIC is an independent consulting firm that is not affiliated with, endorsed by or sponsored by NDIA, SEI, or any other third party.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 2
World-Class Quality
ObjectivesDescribe some requirements problems from industry.
Present a useful classification of requirements problems.
Describe some practical approaches that real organizations have used to successfully measure their requirements.
Provide some real measurement examples.
Answer any of your questions.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 3
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 4
World-Class Quality
Why Focus on Requirements?
“The hardest single part of building a software system is deciding what to build... No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.”
Fredrick Brooks, Jr.
• Reference: Brooks, Fredrick P., Jr. “No Silver Bullet: Essence and Accidents of Software Engineering”. IEEE Computer, 10-19, April 1987.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 5
World-Class Quality
Why Focus on Requirements?
A research report from the Standish Group highlighted the continuing quality and delivery problems in our industry and identified three leading causes:
• Lack of user input
• Incomplete requirements and specifications
• Changing requirement specifications
• Reference: “Chaos”, Compass, The Standish Group, 1997, used with permission.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 6
World-Class Quality
Problems with RequirementsProblems of requirements elicitation can be grouped into 3 categories:
1. Problems of Scope: the requirements mayaddress too little or too much information.
2. Problems of Understanding: problems within groups as well as between groups such as users and developers.
3. Problems of Volatility: the changing nature of requirements.
• Reference: Christel, Michael G. and Kang, Kyo C. “Issues in Requirements Elicitation”, CMU/SEI-92-TR-12, 1992.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 7
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 8
World-Class Quality
Engineering Process Areas
RD PI
VAL
CustomerTS
VER
REQM Requirements
Customer needs
Product and product component requirements
Product components, work products, verification and validation reports
Productcomponents
Alternativesolutions
Require-ments
Product
• Reference: “Capability Maturity Model® Integration (CMMISM), Version 1.1”, CMU/SEI-2003-TR-011, March 2003
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 9
World-Class Quality
Requirements DevelopmentSG 1: Develop Customer Requirements:
SP 1.1-1: Collect Stakeholder NeedsSP 1.1-2: Elicit NeedsSP 1.2-1: Develop the Customer Requirements
SG 2: Develop Product Requirements:SP 2.1-1: Establish Product and Product-Component Requirements SP 2.2-1: Allocate Product-Component Requirements SP 2.3-1: Identify Interface Requirements
SG 3: Analyze and Verify Requirements:SP 3.1-1: Establish Operational Concepts and ScenariosSP 3.2-1: Establish a Definition of Required Functionality SP 3.3-1: Analyze Requirements SP 3.4-3: Analyze Requirements to Achieve BalanceSP 3.5-1: Validate RequirementsSP 3.5-2: Validate Requirements with Comprehensive Methods
• Reference: “Capability Maturity Model® Integration (CMMISM), Version 1.1”, CMU/SEI-2003-TR-011, March 2003
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 10
World-Class Quality
Requirements ManagementSG 1: Manage Requirements:
SP 1.1-1: Obtain an Understanding of the Requirements
SP 1.2-2: Obtain Commitment to Requirements
SP 1.3-1: Manage Requirements Changes
SP 1.4-2: Maintain Bidirectional Traceability of Requirements
SP 1.5-1: Identify Inconsistencies between Project Work and Requirements
• Reference: “Capability Maturity Model® Integration (CMMISM), Version 1.1”, CMU/SEI-2002-TR-011, March 2002
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 11
World-Class Quality
CMMISM and RequirementsRequirement processes need to be defined, trained, and improved (e.g., OPF, OPD, OT, OID).
Support processes are critical for measuring and managing requirements (e.g., CM, MA, PPQA).
Defects need to be removed and prevented in requirements (e.g., PI, VER, VAL, CAR).
IPPD also contains allocating requirements to teams (e.g., IPM for IPPD).
Supplier Sourcing requires managing supplier requirements.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 12
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 13
World-Class Quality
Practical Metric Approaches1. Define the requirements process using best
practices and measurement in mind (e.g., CMMISM, SW-CMM®, ETMX, etc).
2. Use CM and track number of CR/PRs to the requirements baselines. Also measure requirements volatility.
3. Use basic requirements metrics (e.g., priority, stability, risk, number of requirements, etc).
4. Use inspections/peer reviews and track defect metrics (e.g., defect types).
5. Use operational definitions to make requirements measurable.
6. Use the QIC Process Measurement FrameworkSM.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 14
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 15
World-Class Quality
1. Example Requirements Process Based on CMMISM
DevelopRequirements
ManageRequirements
• Updated Customer Requirements
• Customer Requirements
• Product Requirements
• Change Requests
• Problem Reports
• Updated Product Requirements
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 16
World-Class Quality1. Manage Requirements Process
Purpose: Effectively Manage Req. Changes
Roles: Project Manager (PM), CCB
Inputs OutputsTasksEntry eXit
MEASUREMENT
• Product Req.
• Change Requests• Problem Reports
Cust Req.Prod Req.InspectedANDBaselinedORCR/PR isOpen
1. PM checksfor new CR/PRs2. Bring CR/PRsto CCB3. Use CM process
• CR/PRsareResolvedANDCust Req.Prod Req.InspectedANDUnder CM
• # of CR/PRs• Req. Volatility
• Customer Req.
• Customer Req.
• Product Req.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 17
World-Class Quality
2. Use CM Metrics
RequirementsBaseline
ImplementationBaseline
ProductBaseline
• Place the requirements under formal CM and use CCB’s to control changes.
• Measure change requests and problem reports (e.g., totals, trends, states, etc).
• Can also measure requirements volatility.
Fundamental Baselines
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 18
World-Class Quality
3. Example Requirement Metrics
Priority(H/M/L)
Risk(H/M/L)
Stability(H/M/L)Allocation
Reference(e.g.,
customer)Requirement#
2
System shall send an RTF FAX
1
Aircraft position shall be updated by the Inertial Navigation System (INS) Solution
SOW # 10-20.3 Software H L M
ORD #2-30-20.3.4.4
INS SubsystemTeam
M M H
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 19
World-Class Quality
4. Example Requirements Checklist Categories
1. Clarity 2. Completeness 3. Complexity 4. Consistency 5. Constraints 6. Feasibility 7. Functionality/Logic 8. Interfaces 9. Standards 10. TBDs 11. Testability 12. Traceability Etc.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 20
World-Class Quality
5. Example Operational Definition
What is a good requirement? When is a requirement defined? Questions like these are difficult to answer without operational definitions.
An operational definition precisely and concisely defines a measurable requirement that states:
• What it has to do
• How well it has to do it
• Under what conditions it has to do it• “Adapted from “Requirements Capture and Evaluation Process” Training Notebook, Lockheed Martin Space Information Systems, NASA Shuttle Onboard Software, 1996.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 21
World-Class Quality
5. Example Operational Definition
2
CalculatePercentage to 3 decimal places
Students that pass first test by => 70% score
1
BaseMeasure
Lowerlimit
UpperLimitConditionsRequirement
(What)#
PercentPlus or minus .001
Percent
Report total percentage of students that passed the first test and graduatedReport total percentage of students that failed the second test and did not graduate
Students that failed second test by < a 70% score
Plus or minus .001
CalculatePercentage to 3 decimal places
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 22
World-Class Quality
6. QIC Process Measurement FrameworkSM
GOALS
CONTROL
PLAN
METRICSKEY QUESTIONS
IMPROVE
DC
• DC = Data Collection; DS = Data Storage
Cost, defects, effort, size, schedule, etc.
Cost, defects, effort, size, schedule, etc.
Cost, defects, effort, size, schedule, etc.
DS
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 23
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 24
World-Class Quality
Some Advanced ApproachesJuran Model: Customer requirements are written in the customer’s language, then translated into the product requirements written in producer’s language. This can work with CMMISM!
QFD/Juran’s Quality Planning Process: Measurable requirements that meet customer needs using a defined process.
Usage Scenarios/Use Cases/Operational Scenarios:Measure requirements based on user needs.
Requirements written in formal languages and use formal metrics.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 25
World-Class Quality
OutlineWhy Focus on Requirements?
CMMISM Requirements Overview
Practical Measurement Approaches
Requirement Measurement Examples
Some Advanced Approaches
Summary
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 26
World-Class Quality
SummaryThe hardest single part of building a system is the requirements.
The top requirements problems are inadequate requirements specifications, changes to requirements, and lack of user input.
Requirements problems fall into problems of scope, understanding, and volatility.
There are practical measurements that you can use today that will help you manage requirements.
Copyright © 1990-2003 by Quality Improvement Consultants, Inc. (QIC) Slide 27
World-Class Quality
Measurably Improving Your Requirements Based on the CMMISM
Tim Olson, PresidentQuality Improvement Consultants, Inc. (QIC)(760) 804-1405Tim.Olson@qic-inc.comwww.qic-inc.com
2003 CMMISM Technology Conference
® CMM is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.SM CMMI is a service mark of Carnegie Mellon University.QIC is an independent consulting firm that is not affiliated with, endorsed by or sponsored by NDIA, SEI, or any other third party.