Copyright © 2001, Software Productivity Consortium NFP, Inc. SOFTWAR E PRODUCTIVITY CONSORTIUM...
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
2
Transcript of Copyright © 2001, Software Productivity Consortium NFP, Inc. SOFTWAR E PRODUCTIVITY CONSORTIUM...
Copyright © 2001, Software Productivity Consortium NFP, Inc.
SOFTWAREPRODUCTIVITYCONSORTIUM
Integrating Information Models:
the essential element of successful measurement
programs
COCOMO Forum October 2004
Chris MillerSoftware Productivity Consortium
2Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Motivation
• Popular emphasis on goal-based measurement, but— Anyone can have goals— Goals often are not consistent— Organizational goals may be independent of
project-specific issues
• Measurement initiatives often implemented as disjoint collections of data
• Measurement systems across the enterprise must be aligned to provide a true enterprise perspective
• Quantification of casual relationships provide the basis for predictive analysis and forecasting performance
3Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Sales
CustomerSatisfaction
Market
INNOVATION
TIME-TO-MARKET
PRICE
QUALITY
Information Needs
Enterprise
Process
Project
Product
4Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Enterprise
Process
Project
Product
Types of Information Needs
Measurement should support business goals and objectives
Information needs flow downward
Data and information are passed upward
Business performance
Efficiency and effectiveness of production
Accomplishment of project budget and schedule
Satisfaction of customer requirements
5Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Enterprise View
• Need to measure business performance—Focusing on strategic goals —Reporting profitability (program, business
unit)—Providing information for marketing
• Enterprise may ask questions such as:—Should we invest in a new product line?
• When present, an enterprise policy for managing and measuring drive measurement behavior and actions
6Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Balanced ScorecardP E R F O R M A N C E I N DI C A T O R S
OU T C O M E ME A S U R E S
( lag indicators of mission success )
OU T P U T ME A S U R E S
( lead indicators of goal achievement )I M P R O V E M E N T
I N I T I A T I V E S
S T R A T E G I C
OB J E C T I V E S
( measures of CSFs & progresstoward ultimate ends )
( measures of progresstoward intermediate ends )
C U S T O M E R A N D O T H E R S T A K E H O L DE R
C1 – Responsiveness to trends andopportunities
Market presence & leadership Name recognition
Number of product imitators
Focus Groups
C2 – Customer retention and growth *
Satisfaction “after the sale” Reduction of complaints “Customers First”
I N T E R N A L B U S I N E S S P R O C E S S
I 1 – Organizational alignment Improved core processes Understanding of value added
Mission-driven technology inserts
Process Reengineering
I2 – Operational competency Ability to provide customer-centered information access
Trained Web developers Corporate Web Site
L E A R N I N G , I N N O V A T I O N , R E N E WA L , A N D M A T U R I T Y
L1 – Astute executive decisions Availability, usability ofstrategic information and aids
Competitive information andexpertise gathered and organized
KnowledgeManagement
L2 – Product/servicedistinctiveness
Innovations fielded Breadth and depth of expertise
Intraorganizational coordination
OrganizationalDevelopment
F I N A N C I A L
F1 – Economic sustainability Expansion into new markets
Exposure to risk
Consortium membership
Joint ventures launched
Business Alliances
F2 – Profitability Return on equity High-leverage-point investments ABM
* or market share
*Source: Robert S. Kaplan, and David P. Norton. “The Balanced Scorecard – Measures that Drive Performance,” Harvard Business Review, January 1992.
7Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Process View
• Concurrent processes exist in a typical enterprise (systems, acquisition, development, operations, logistics, marketing, …)
• Focus on the process capability — Definition of processes
– Including measurement points and vehicles— Adherence to defined processes
– Self-audits— Improvement of processes
– Effectiveness– Efficiency
8Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Product View
• The system itself is the primary product—Component quality characteristics—Need to assess impact of changes across the
system components—Customer satisfaction with product
• Products and deliverables—System architecture, specifications,
software, interface control documents, test cases & procedures, users’ guides, …
9Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
…Applied Across the Life Cycle
Critical process performance measured in process, during process execution
Product quality characteristics measured and addressed at product transition points
throughout the life cycle
De
fec
tsMaintenance PhasesDevelopment Phases
10Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Project Views
• Focus on the activities and tasks as a project—Meeting product delivery dates—Planning effort —Monitoring and controlling effort
– Adherence to management plan—Customer satisfaction with performance
11Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
PSM Analysis Model
Technology Effectiveness
ProcessPerform ance
Product Size and Stability
Resourcesand Cost
Schedule and Progress
ProductQuality
Custom er Satisfaction
12Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Integration of Views
EnterpriseInfo. Needs
ProcessInfo. Needs
ProjectInfo. Needs
ProductInfo. Needs
EnterpriseInfo. Models
ProcessInfo. Models
ProjectInfo. Models
ProductInfo. Models
Systems Eng.Software Eng.
SystemsSoftware
Senior Mgmt Commitment
Pla
n
An
alyz
e
Eva
luat
e
Automate
Common Process
Tailored Support
Level-Specific Information
13Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Integration Activities
1. Each view maintains it’s own set information models for monitoring their own information needs
2. Senior management commitment — Buy-in is needed at the most senior level to
facilitate and drive an integrated measurement program (after all they will benefit the most)
3. Use common base measures— When planning to do measurement use a
standard— Standard consists of common ways to analyze
and evaluate the data and measurement process
4. Build automated tool support around the standard base measures and analysis techniques— Requires tailoring for each view
14Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Step 1: Information Models by View Measurement Information Model
• Defines measurement product
• Links measurable entities to information needs
• Provides rigorous definitions of basic concepts
• Measurement construct consists of :—Base Measures—Derived Measures—Indicators
• Explains prevalence of three-level measurement models (e.g., Goal-Question-Metric [GQM])
Information Model
MeasurementConstruct
Entities
Information Needs
InformationProduct
MeasurableConcept
Attribute
15Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
INFORMATION NEEDS
ANALYSIS RESULTS
ANALYSIS RESULTS AND
PERFORMANCEMEASURES
IMPROVEMENTACTIONS
Adapted from ISO/IEC CD 15939, Information Technology – Software Measurement Process
USER FEEDBACK
EstablishCommitment Plan Evaluate
Technical & Management
Processes
Core Measurement Process
ExperienceBase
MEASUREMENTPLAN
Perform
Database
Measurement Process Model
16Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Step2: Buy-in with Business Case
Engineering ManagementProject ManagementExecutive Management
Business Goals
QualityCustomer Satisfaction
Cycle TimeCost
Reduced Rework
Fewer Failures
Lower Price
Lower support costs
Productivity
Increased Capacity
Reduced Effort
Market ShareProfit
Shorter schedules
Time-to-market
Higher MarginLow-cost provider
Retention Referrals
Award fees Bonuses
Image
SPI Activities
17Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Map Goals/Measures/Activities (Cycle Time Example)
Quality
Cycle Time
Reduced Rework
Productivity
Increased Capacity
Process Engineering
Defined Process
Defined Process
Efficient Process
Predictable Schedules
Better Estimates
Stable Requirements
Historical Data
Estimating Model
Measurement Program
Change Control
Requirements Management Defined
Process
Shorter Schedules
(Cust. Sat.)
Reduced Labor (Cost)
Early to Market
Process Mapping
Process Initiatives
Desired Results of Activities
Fewer Steps, Delays
Business Goals,
Objectives
Engineering Analysis, Response
18Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Step 2 Exit Criteria
• Business case needs to:—Obtain senior management commitment —Show information flow down—Highlight key activities
• Provide business-oriented information for each view—Potentially new information needs—Highlight linkage to other views
19Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Step 3: Unify Definitions
• Base measures need to remain consistent across all measurement activities:—Uniform base measures—Data formats—Identify common entities and attributes
• Analysis techniques are tailored to generate unique indicator base on individual information needs:—Define derived measures—Consistent analysis techniques—Identify common reporting standards and
graphic for related indicators
20Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Uniform Base Measures
• Consistent use of —Data labels (e.g.,
SLOC)—Data formats
• Base measures are the lowest common denominator—Validate across all
information models
• Avoid redundant data collection methods
• Maintain integrity of scales and units
Base Measure NameRelease 01.03 Lines of code
AttributeC++ language statements
Measurement MethodCount semicolons in all program units
Type of Measurement MethodObjective
Scale (including precision)Whole numbers from 0 to infinity
Type of ScaleRatio
Unit of MeasureLine of code
Base Measure Definition/Template
21Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Uniform Analysis
• Mitigate redundant naming conventions between indicators — (e.g., Productivity)
• Set standard analysis approach, techniques, and methodologies
• Ensure decision criteria in consistent across information models (and views)
• Use consistent graphic representations for similar data
IDCost Performance Index for Project XX
Information NeedEnter ID of information need
Measures UsedDerived Measure(s)Cost Performance Index (CPI)Base Measures1.Actual Cost of Work Performed2.Budgeted Cost of Work Performed
Function(s)
Calculate CPI:
For any given project, perform the following calculation:
Actual Cost of Work Performed (ACWP)Budgeted Cost of Work Performed (BCWP)
Analysis ModelSimple CPI calculation. Optimal performance is to keep actuals close to or at the center line value of 1.0.
Decision CriteriaExpected value is ‘1.0’. Decision needed if actual value is +/– 10%.
Graphical Representation.Standard CPI line graph with value of 1 represented as a horizontal line, positive values above and negative values below.
CPI =
Indicator Definition/Template
22Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Step 4: Tool Support
• Automated tools support is required to handle the mass of data collection and analysis associated with an integrated measurement program across many views
• Base measure definitions and analysis techniques provide a basis for selection of tool suite
• Understanding the information flow in the business case helps during trade-off decisions among different tools
23Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Summary• An effectively designed measurement program must
consider the interrelationships of the views
• An integrated measurement program will need to establish:— Uniform base measures— Common tools and databases
• Each view uses the base measures to:— Define indicators
• Allows for the flexibility for each view to address their information needs
• Integration is an impossible task without senior management support
24Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Back-up Slides
25Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
Interpretation Estimate or evaluation that provides a basis for decision making
Indicator
Model
DerivedMeasure
Value resulting from applying the algorithm to two or more measures
Algorithm combining measures and decision criteria
DerivedMeasure
Operations mapping an attribute to a scaleMethod
BaseMeasure
Function Algorithm combining two or more base measures
Value resulting from applying the method to one attribute
AttributeAttributeProperty relevant to information needs Entities
Information NeedsInformation
Product
Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework
Measurement Construct
Method
BaseMeasure
26Copyright © 2004, Software Productivity Consortium NFP, Inc.
Integrating Information Models
0.5 > Organization average of 0.3 defects per page Indicator
Model
DerivedMeasure
Compare defect density to SE organizational data
InterpretationCompare defect density to organizational average, perform causal analysis if > 1.0 defects per page
InformationProduct
Indicator summarized for all SE documents in monthly status
Measurement Construct Example
DerivedMeasure
FunctionDivide (defects per page)
0.5 defects per page
AttributeAttributeEntities
Count only bullets in minutes
Method
BaseMeasure
50 defects found
Defects per peer review
Method
BaseMeasure
Count only technical pages
100 pages reviewed
Pages reviewed
SE DocumentPeer Review
Information NeedsSystems Engineering performance – documentation quality?
Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework