Process Improvement for Software Projects CMMI and Lean Six Sigma USC CSC1 510 16 Nov 2015 Rick...
-
Upload
cory-evans -
Category
Documents
-
view
215 -
download
0
description
Transcript of Process Improvement for Software Projects CMMI and Lean Six Sigma USC CSC1 510 16 Nov 2015 Rick...
Process Improvement for Software Projects
CMMI and Lean Six Sigma
USC CSC1 51016 Nov 2015
Rick Hefner, Ph.D.California Institute of Technology
2
Agenda
• Current Challenges Facing the Software Industry
• Capability Maturity Model Integrated
• Lean Six Sigma
• Corporate Support
What are some of the challenges facing software companies?
3
Software Projects Have Historically Suffered from Mistakes
Reference: Steve McConnell, Rapid Development
People-Related Mistakes 1. Undermined motivation 2. Weak personnel 3. Uncontrolled problem employees 4. Heroics 5. Adding people to a late project 6. Noisy, crowded offices 7. Friction between developers and customers 8. Unrealistic expectations 9. Lack of effective project sponsorship 10. Lack of stakeholder buy-in 11. Lack of user input 12. Politics placed over substance 13. Wishful thinking
Process-Related Mistakes14. Overly optimistic schedules 15. Insufficient Risk Management16. Contractor failure Insufficientplanning 17. Abandonment of planningunder pressure 18. Wasted time during the fuzzy front end 19. Shortchanged upstreamactivities 20. Inadequate design 21. Shortchanged qualityassurance 22. Insufficient managementcontrols 23. Premature or too frequentconvergence 25. Omitting necessary tasks from estimates 26. Planning to catch up later27. Code-like-hell programming
Product-Related Mistakes28. Requirements gold-plating 29. Feature creep 30. Developer gold-plating 31. Push me, pull me negotiation32. Research-orienteddevelopment
Technology-Related Mistakes 33. Silver-bullet syndrome 34. Overestimated savings fromnew tools or methods 35. Switching tools in the middleof a project 36. Lack of automatedsource-code control
Standish Group survey of 13,000 projects (2003)• 34% successes• 15% failures• 51% overruns4
Top Software Engineering Issues in Defense Industry
1. The impact of requirements upon software is not consistently quantified and managed in development or sustainment
2. Fundamental system engineering decisions are made without full participation of software engineering
3. Software life-cycle planning and management by acquirers and suppliers is ineffective
4. The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry
5. Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems
6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments
7. Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk
5
“Top Software Engineering Issues In Defense Industry”, NDIA Systems Engineering Division and Software Committee, Sep 2006
Many Approaches to Solving the Problem
• Which weaknesses are causing my problems?• Which strengths may mitigate my problems?• Which improvement investments offer the best return?
People
Product
Technology
Tools
ManagementStructure
BusinessEnvironment
Process
Methods
6
7
Heritage of Standards for Systems Engineering
EIA / IS 632
ISO/IEC 15288
Mil-Std-499BMil-Std-
499A
1994
1994
1994
19982002
1974
(Not Released)Mil-Std-499
1969
(Trial Use)IEEE 1220
1998
(Full Std)
LegendSupersedesSource for
EIA632
1998
EIA/IS 731
SE CM
IEEE 1220
(Full Std)(Interim Standard)
2002
CMMI-
SE/SW/IPPD
(Interim Standard)
(FDIS)
ISO/IEC 19760
2002
(PDTR)
2002
ISO/IEC 15504(FDIS)
Standards for Systems Engineering, Jerry Lake, 2002
8
The Frameworks QuagmireSarah A. Sheard, Software Productivity Consortium
Evolution of the Frameworks Quagmire, Software Magazine
9
Two Complimentary Approaches to Process Improvement
Data-Driven (e.g., Lean Six Sigma)
• Clarify what your customer wants (Voice of Customer)
– Critical to Quality (CTQs)• Determine what your processes
can do (Voice of Process)– Statistical Process Control
• Identify and prioritize improvement opportunities
– Causal analysis of data• Anticipate your customers/
competitors (Voice of Business)– Design for Six Sigma
Model-Driven (e.g., CMMI)
• Determine the industry best practice
– Benchmarking, models• Compare your current practices to
the model– Appraisal, education
• Identify and prioritize improvement opportunities
– Implementation– Institutionalization
• Look for ways to optimize the processes
Capability Maturity Model Integrated
10
How would you ensure best practices are implemented consistently in your
software company?
12
What is the Capability Maturity Model Integrated?
• The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management
– Developed under the sponsorship of DoD– Consistent with DoD and commercial standards
Three Constellations sharing common components and structure
• CMMI for Development - used by engineering organizations
• CMMI for Acquisition - used by buyers (e.g., govt. agencies)
• CMMI for Services - used by service providers (e.g., help desk)
13
Basic Building Blocks – 22 Process Areas
Project Management• Project Planning• Project
Monitoring and Control
• Supplier Agreement Management
• Integrated Project Management)
• Risk Management
• Quantitative Project Management
Engineering• Requirements
Development• Requirements
Management• Technical
Solution• Product
Integration• Verification• Validation
Support• Configuration
Management• Process and
Product Quality Assurance
• Measurement and Analysis
• Decision Analysis and Resolution
• Causal Analysis and Resolution
Process Management
• Organizational Process Focus
• Organizational Process Definition
• Organizational Training
• Organizational Process Performance
• Organizational Performance Management
Implemented byeach project
Implemented bythe organization
14
Expected Practices Provide Guidancefor Implementation and Institutionalization
SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product
and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost
SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan
SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment
GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational PolicyGP 2.2 Plan the ProcessGP 2.3 Provide ResourcesGP 2.4 Assign ResponsibilityGP 2.5 Train PeopleGP 2.6 Manage ConfigurationsGP 2.7 Identify and Involve Relevant
StakeholdersGP 2.8 Monitor and Control the ProcessGP 2.9 Objectively Evaluate AdherenceGP 2.10 Review Status with Higher Level
ManagementGG 3 Institutionalize a Defined Process
GP 3.1 Establish a Defined ProcessGP 3.2 Collect Improvement Information
Project Planning – Implementation Project Planning - Institutionalization
15
Practice Ratings for the Organization/Projects
16
How is the CMMI Used for Process Improvement?
www.sei.cmu.edu/ideal/
IDEAL Model
17
Typical CMMI Benefits Cited in Literature
• Reduced costs– 33% decrease in the average
cost to fix a defect (Boeing)– 20% reduction in unit
software costs (Lockheed Martin)
• Faster Schedules– 50% reduction in release
turnaround time (Boeing)– 60% reduction in re-work
following test (Boeing)
• Greater Productivity– 25-30% increase in productivity
within 3 years (Lockheed Martin, Harris, Siemens)
• Higher Quality– 50% reduction of software
defects (Lockheed Martin)
• Customer Satisfaction– 55% increase in award fees
(Lockheed Martin)
Lean Six Sigma
18
19
What is Lean Six Sigma (LSS)?
• Lean Six Sigma is a powerful approach to improving business
• LSS improvement projects are performed by teams
• Teams use a set of tools and techniques to understand problems and find solutions
• Lean Six Sigma integrates tools and techniques from two proven process improvement methods
+
DMAIC: Define-Measure-Analyze-Improve-Control
A proven, structured problem-solving approach for improving existing processes
• Define: Identify the issue causing the perceived problem or improvement opportunity
• Measure: Collect data from the process
• Analysis: Study the data for cause and effect
• Improve: Eliminate cause(s) and determine resulting effects
• Control: Sustain the gains by monitoring the system
20
21
DMAIC Toolkit
• Charter• Kano Model• Voice of the
Customer• Voice of the
Business• Quality
Function Deployment
• Voice of the Process
• Data Collection Methods
• Measurement System Evaluation
Statistical Controls • Control
Charts• Time Series
methodsNon-Statistical Controls• Procedural
adherence• Performance
Management• Preventive
measures
• Design of Experiments
• Modeling• ANOVA• Tolerancing• Robust Design• Systems
Thinking• Decision & Risk
Analysis
• Cause & Effect Diagrams/ Matrix
• Failure Modes & Effects Analysis
• Statistical Inference
• Reliability Analysis
• Root Cause Analysis, including 5 Whys
• Hypothesis Test
Define Measure Analyze Improve Control
22
Lean - Definition
• Lean is the philosophy of improving performance by reducing waste and eliminating bottlenecks
• Used together, the two methodologies become a powerful set of tools for improving business processes "The 8 Wastes - DOWNTIME" by
Source (WP:NFCC#4). Licensed under Fair use via Wikipedia.
23
Organizational Adoption: Roles & Responsibilities
• Champions – Facilitate the leadership, implementation, and deployment
• Sponsors – Provide resources
• Process Owners – Responsible for the processes being improved
• Master Black Belts – Serve as mentors for Black Belts
• Black Belts – Lead major Six Sigma projects– Typically requires 4 weeks of training
• Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt
– Typically requires 2 weeks of training
24
A Typical Lean Six Sigma Project in Aerospace
The organization notes that systems integration has been problematic on past projects (budget/schedule overruns)
A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s)
The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test
Procedures and criteria for better peer reviews are written, using best practices from past projects
A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem
The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects
Corporate Support
25
26
Organizational Infrastructure Required
Policies, Processes,Templates & Tools
Best-Practice Libraries
Process Group
Audits & AppraisalsMeasurement RepositoriesPredictive Modeling
Developing and maintaining mature processes requires significant time and investment in infrastructure
Process ImprovementTraining Program
Communications
0
5
10
15
20
25
1 11 21 31 41 51 61 71
UCL
_X
Defe
cts
per c
ompo
nent
Component #
0
5
10
15
20
25
1 11 21 31 41 51 61 71
UCL
_X
Defe
cts
per c
ompo
nent
Component #
27
Northrop Grumman Approach:Institutionalizing Our Improvements
InternalBest Practices
ISO/AS9100 Findings
CMMI Appraisal Findings
Policy
Procedures
Process
eToolkit PAL
WorkbenchStartIt! My MS Portal
Tools
Information
Checklists and Guides
Templates and Examples
Disposition
Independent Audits
• Systems/ Software Engineering Process Group
• Working Groups• Program
Management Advisory Board
Lessons Learned & Metrics
Analysis
Customer Comments
Configuration Control Board
ExternalBest Practices
Industry Standards
Six Sigma Projects
We systematically analyze quality and process data and trends to determine
how to improve our processes
We improve our process assets based on internal and external best
practices
Deployed to
programs
Increasing program
efficiency
msCAS
PCDB
28
Lessons Learned
• Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a goal”
– Reinforces that change (improvement) is a way of life
• Benefits result from institutionalizing local improvements across the wider organization
– CMMI establishes the needed mechanisms
• Training over half the staff as Lean Six Sigma Green Belts resulted in a change of language and culture
– Voice of Customer, data-driven decisions, causal analysis, etc.– Better to understand/use tools in everyday work than to adopt the “religion”
• CMMI and Lean Six Sigma compliment each other– CMMI can yield behaviors without benefits– Lean Six Sigma improvements based solely on data may miss innovative improvements
(assumes a local optimum)