The Importance of Software Process Management for Software ...
“The Importance of Software Process Management for ... · “The Importance of Software Process...
Transcript of “The Importance of Software Process Management for ... · “The Importance of Software Process...
�The Importance of Software Process Management for Software Quality, Testing and
Industry Development�Presented at the
China�s Software Quality, Testing and Industry Development Strategy Seminar
China's Quality Seminar 1
Development Strategy SeminarBeijing October 23, 2004
Angel G. JordanUniversity Professor Emeritus
Provost EmeritusCarnegie Mellon University
Founder, Software Engineering Institute
Issues at the Seminar
nSoftware Quality
nScope of Testing
China's Quality Seminar 2
Scope of Testing
nRequirement Analysis
What I will cover in this presentation
n The Significance of Software Qualityn The Importance of Software Testingn Quality in Delivering Offshore IT Servicesn State of Practice Versus SEI’s Vision in Software Engineeringn Software Process Management
China's Quality Seminar 3
n Software Process Managementq CMMIq Capability Levelsq Maturity Levelsq How CMMI can help an organization
n Conclusions
Software Quality Assurance
n Defining Software Quality Assuranceq Monitoring the software development processq Complying with standards and accepted methods
China's Quality Seminar 4
q Complying with standards and accepted methodsq Anticipating errors --- prevention
Software Quality Assurance�
nComputer system failures are caused by software defects
nDefects caused not only by Programming errors, but byqNot clear application requirements or changing requirements
nSoftware complexity may also induce errors
China's Quality Seminar 5
nSoftware complexity may also induce errorsnOther causes of errors may be:
qTime pressure because of unrealistic schedulesqNot well documented codeqDefective and/or poorly documented development tools
Introducing Software QA processes in an organization
n In large organizations involving high risk projects formalized QA processes are mandatory.
n For lower risks, QA processes are to be balanced with productivity.
n For small groups or projects, ad-hoc processes may
China's Quality Seminar 6
n For small groups or projects, ad-hoc processes may be appropriate, depending on customers and projects.
n Dependence on:q Team leaders or managersq Feedback to developersq Communication between customers, managers, developers, and testers
Significant payoff from investment in QA comes from:
n Processes in management of requirements aiming atq Testable requirement specifications as part of requirements or design documentation
China's Quality Seminar 7
q Or in agile-type environments coordination with end users
q Inspections of the design and codeq Evaluation of past projects
Management can play a key role in:
n Recognizing:q The importance of Verification and Validationq The use of Walkthrough and Inspectionq Different testing methods to be considered
China's Quality Seminar 8
q Different testing methods to be consideredq An understanding of the typical problems arising in the software development process and n An appreciation of potential solutions
q An understanding of the true meaning of Software Quality and what it entails
Software Quality entails:n Recognition that Quality Software is:
q Reasonably bug-free, q delivered on time and within budget, q meets requirements and/or expectations, q and can be maintained
n Quality software incorporates
China's Quality Seminar 9
Quality software incorporatesq Good codeq Good design
n The vision of the SEI regarding Quality emphasizes:q Software Process Management, in order to: q Minimize Testing
n We will talk about this later
The Software Life Cyclen Starts when application is conceived and ends when it is no longer in use. It includes:q Initial conceptq Requirements analysisq Functional designq Internal designq Documentation planningq Test planning
China's Quality Seminar 10
q Test planningq Codingq Document preparationq Integrationq Testingq Maintenanceq Updatesq Retestingq Phase-out
Software Testing
n Defining Software Testingq Testing involves operation of a system or application under control conditions and evaluating the results
China's Quality Seminar 11
q Controlled conditions should be both normal and abnormal
q Testing should attempt to make things go wrongq The emphasis is in detection
Software Testing Engineeringn The role of the Software Test engineern The role of the Software QA engineern The role of the QA or Test managern The role of Documentation in QAn The importance of Requirementsn Developing and running software tests
China's Quality Seminar 12
n Developing and running software testsq The test planq The test case
n Actions needed after defects are foundn Configuration management
n Actions when software has many defects
Considerations in Testing
n The end of the testing processn Time allotted for comprehensive testingn The size of the project and extensive testingn Coping with changing requirements When the application has functionality that was
China's Quality Seminar 13
n When the application has functionality that was not in the requirements
n Software QA processes vis-a-vis productivityn Coping with fast growth and QA processes
Other considerations in Testing
n Client/Server Environments and Testing
n Testing World Wide Web sites
n Testing and object-oriented designs
China's Quality Seminar 14
n Testing and object-oriented designs
n Extreme Programming and Testing
Reference for QA and Testing: http://www.softwareqatest.com/
Software QA and Testing Resource Center
n http://www.softwareqatest.com/
China's Quality Seminar 15
n http://www.softwareqatest.com/
Software Quality Engineering
n http://www.sqe.com/index.aspn Collocated with STARWEST 2004.n Sponsors useful courses
Software Testing Foundations course
China's Quality Seminar 16
q Software Testing Foundations courseq Certification training courseq Writing Testable Requirementsq Essential Software Requirements
n Balancing Plan-Driven and Agile Approaches
Quality in Delivering Offshore IT Services
n The Delivery of Offshore Services n Quality Framework and its Business Benefits
q Organizational Effectivenessq Service delivery
China's Quality Seminar 17
q Service deliveryq Quality Certifications:
n CMM, CMMI, Six Sigma
n Reference: Stephanie M. Torto Ruthbea Yesner, ‘The Role and Benefits of Quality in Delivering Offsore IT Services’, December 2003, White Paper, Sponsored by Wipro Technologies.
Quality influences Business Results
n Organizational Effectivenessq Commitment to qualityq Highly skilled employeesq Responsiveness and flexibilityStrong work ethic
China's Quality Seminar 18
q Strong work ethicq Lower turnover of employeesq Robust project management processesq Strong communication flow between client and services firm
Outcomes-Benefits to the customer
n Lower maintenance costn Lower schedule overrun costn Lower development cost
n IDC Recommendations * q Separate the forest from the trees
China's Quality Seminar 19
q Separate the forest from the treesq Deepen understanding of organizational effectivenessq Quantify the benefits to the businessq Use a balance of onsite and offshore resources
* IDC is a leading provider of data driven research analysis http://www.idcindia.com/ http://www.idc.com.cn/ http://www.idc.com.cn/eindex.php
The Importance of Quality Certifications
n The Capability Maturity Model (CMM)n The People Capability Maturity Model (PCMM) n The Capability Maturity Model Integrated
China's Quality Seminar 20
(CMMI)q We will talk about these later
n Six Sigma ISO, IEEE, ANSI Standards
SEI Strategic ThemesnPredictably better, faster, and cheaper by -
nMoving to the leftnEmbrace a systems engineering approach and make better decisions before coding to predictably improve quality, cost, and schedule.
China's Quality Seminar 21
nReusing everythingnReuse code, but also the architecture and knowledge from building similar systems.
nNever making the same mistake twicenLeverage lessons learned.
State of Practice Versus SEI’s Vision
Development Integration and System Test
Software state of practice (“test in” quality)
60 - 80 % of effort and cost
China's Quality Seminar 22
* move to the left !
* reuse everything
* never make the samemistake twice*Ref: Standish Group, www.standishgroup.com, 1999
Development Integration and System Test
World-class developers “design in” quality
SEI Technical Program
Capability MaturityModel
Integration
Integration Software Intensive Systems
Team Software Process
Survivable Systems
Product
The right software delivered defect free, on cost, on time, every time
High confidence, evolvable,product lines
with predictable and improvedcost, schedule, and quality
China's Quality Seminar 23
Management PracticeInitiatives
Integration
Acquisition Support Systems
Systems
Performance Critical Systems
Software Architecture Technology
Technical PracticeInitiatives
Software Engineering Measurement & Analysis
Product Line Practice
Predictable Assembly with
Certifiable Components
General Definition of Process
nA process is a set of practices performed to achieve a given purpose; it may include tools, methods, materials, and/or people.
China's Quality Seminar 24
tools, methods, materials, and/or people.
Quality Leverage Points
nWhile process is often described as a leg of the process-people-technology triad, it may also be considered the “glue” that unifies the other aspects.
Everyone realizes the importance of having a motivated, quality work force but even our finest
PEOPLE
China's Quality Seminar 25
force but even our finest people can’t perform at their best when the process is not understood or operating “at its best.”
n Major determinants of product cost, schedule, and quality
PROCESSTECHNOLOGY
Why Focus on Process?nProcess provides a constructive, high-leverage focus...
nas opposed to a focus on people n The experience and training of your work force is not always enough.
China's Quality Seminar 26
n Working harder is not the answer.n A well defined process can provide the means to work smarter.
n Shifts the “blame” for problems from people to the process
q as opposed to a focus on technologyn Technology, by itself, will most likely not be used effectively.
n Technology, in the context of an appropriate process roadmap, can provide the most benefit.
The Process Management Premise
nThe quality of a system is highly influenced by the quality of the process used to acquire, develop, and maintain it.
nThis premise implies a focus on processes as
China's Quality Seminar 27
nThis premise implies a focus on processes as well as on products.q This is a long-established premise in manufacturing.
q Belief in this premise is visible worldwide in quality movements in manufacturing and service industries (e.g., ISO standards).
Process Improvement Involves Dealing with Multiple Dimensions at One Time
OrganizationStructure
Technical
Strategic??
China's Quality Seminar 28
CMMI based improvement is Much More than simply askingpeople to use a different template.
Managerial Social/Cultural
Strategic
ISOISO
CMMICMMIQAQA
??
Simple Improvement Processes
nDetermine where you want to be.nMake a plan.nExecute the plan.nLearn lessons and do it again.
China's Quality Seminar 29
nAnother improvement process is Plan Do Check Act.
nIDEALSM
nDetermine where you are.
SetContext
BuildSponsorship
CharterInfrastructure
Pilot/Test
RefineSolution
ImplementSolution
AnalyzeandValidate
ProposeFutureActions
Stimulus for Change Acting
Learning
The IDEALSM Model
China's Quality Seminar 30
CharacterizeCurrent &Desired States
DevelopRecommendations
SetPriorities Develop
Approach
PlanActions
CreateSolution
Pilot/TestSolution
Initiating
Diagnosing
EstablishingSM IDEAL is a service mark of Carnegie Mellon University.
Documented Results of CMM-based PI
q Process improvement benefits fall into one or more of these general categories:n improved schedule and budget predictabilityn reduction of reworkn improved cycle timen increased productivity
China's Quality Seminar 31
n improved quality (as measured by defects)n increased customer satisfactionn improved employee moralen increased return on investmentn decreased cost of quality
nAll this is in addition to meeting your customer’s requirement....
CMMI Transition Status- 234 Introduction to CMMI instructors trained- 402 SCAMPI Lead Appraisers trained- 830 students trained in Intermediate Concepts of CMMI
- 179 Introduction to CMMI instructors authorized- 16,161 students trained in Introduction in CMMI
China's Quality Seminar 32
- 16,161 students trained in Introduction in CMMI- 293 SCAMPI Lead Appraisers authorized- 80 active transition partners that offer Introduction to CMMI training
- 147 active transition partners that offer SCAMPI appraisal services
Number of Appraisals Conducted by YearReported as of 2 March 2004
500
600
700
800
China's Quality Seminar 33
0
100
200
300
400
198719881989199019911992199319941995199619971998199920002001200220032004
SPA SEI CBA IPI SCAMPI vX
What is CMMI?
nCMMI q is not a process; does not tell you How to do your work
q is a model of best practices in systems and
China's Quality Seminar 34
q is a model of best practices in systems and software development and in product development
qdoes tell you What to doq is based on the process management premise
CMMI Model Representations
An organization may choose an approach to process improvement from either of the following:process area capability
China's Quality Seminar 35
qprocess area capabilityqorganizational maturity
CMMI models support each approach with a representation.qprocess area capability - continuous representationqorganizational maturity - staged representation
Continuous Representation
•Some benefits of choosing the continuous representation are
• It allows you to select the order of improvement that best meets your organization’s business objectives and mitigates your organization’s
China's Quality Seminar 36
objectives and mitigates your organization’s areas of risk.
• It enables comparisons across and among organizations on a process area by process area basis.
• It provides an easy migration from EIA/IS-731 to CMMI.
PA PA
Process Area Capability
PA
ProjectManagement
Process AreasCategory
Project PlanningProject Monitoring and ControlSupplier Agreement ManagementIntegrated Project Management for IPPDRisk ManagementIntegrated TeamingIntegrated Supplier Management
Organizational Process FocusOrganizational Process DefinitionOrganizational TrainingOrganizational Process PerformanceOrganizational Innovation and Deployment
ProcessManagement
China's Quality Seminar 37
Requirements ManagementRequirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidation
Engineering
Configuration ManagementProcess and Product Quality AssuranceMeasurement and AnalysisDecision Analysis and Resolution Organizational Environment for Integration Causal Analysis and Resolution
Support
Integrated Supplier ManagementQuantitative Project Management
Capability Levels
nCapability levels, used in the continuous representation, describe a sequential order for approaching process improvement within each process area.
nCapability levels build on each other, providing a
China's Quality Seminar 38
nCapability levels build on each other, providing a recommended order for approaching process improvement within a selected process area.
nCapability levels enable you to track, evaluate, and demonstrate your organization’s progress as you improve processes associated with a process area.
Staged Representation
•Some benefits of choosing the staged representation are
• It provides a proven sequence of improvements, each serving as a foundation for the next.
• It permits comparisons across and among organizations by the use of maturity levels.
China's Quality Seminar 39
the use of maturity levels.
• It provides an easy migration from the SW-CMM to CMMI.
• It provides a single rating that summarizes appraisal results and allows comparisons among organizations.
Organizational Innovation and DeploymentCausal Analysis and Resolution5 Optimizing
4 Quantitatively Managed
3 Defined
ContinuousProcess Improvement
QuantitativeManagement
ProcessStandardization
Organizational Process PerformanceQuantitative Project Management
Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidationOrganizational Process FocusOrganizational Process Definition
Higher Quality ProductivityLower Risk
Process Areas for SE/SW/IPPD/SSLevel Focus
China's Quality Seminar 40
3 Defined
2 Managed
Standardization
BasicProjectManagement
Organizational Process DefinitionOrganizational Training Integrated Project Management for IPPDRisk ManagementIntegrated TeamingIntegrated Supplier ManagementDecision Analysis and ResolutionOrganizational Environment for Integration
Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement ManagementMeasurement and AnalysisProcess and Product Quality AssuranceConfiguration Management
Higher RiskRework1 Initial
Maturity Levels-1
nMaturity levels, used in the staged representation,organize selected process areas into five evolutionary plateaus to support and guide process improvement across the organization.
China's Quality Seminar 41
nMaturity levels are defined evolutionary plateaus of process improvement, in which each level is supported by the characteristics of the process areas implemented within the lower levels.
nMaturity levels represent a process improvement evolution for the entire organization.
Maturity Levels-2
Process measuredand statistically controlled
Focus on continuousimprovement
Optimizing
QuantitativelyManaged
Optimizing
4
5
China's Quality Seminar 42
Process unpredictable, poorly controlled and reactive
Process characterized for projects and is often reactive
Process characterized for the organization and is proactive
Defined
Initial
Managed
Defined3
2
1
Staged
ML3
ML4
ML5
Continuous. . .for a single process areaor a set of process areas
Process Area Capability
China's Quality Seminar 43
ML 1
ML2
ML3
. . .for a specified set ofprocess areas across anorganization
PA PA
Process Area Capability
PA
Comparing the Different Representations
n Both representations provide ways of implementing process improvement to achieve business goals.
China's Quality Seminar 44
n Both representations provide essentially the same content but are organized in different ways.
n There is nothing that requires that you use one or the other representation – you can use them bothconcurrently if that suits your business needs.
Representation Synergyn The continuous representation provides additional de factogranularity in support of a staged-focus initiative:• as a guide for detailed tactical planning• to demonstrate intermediate progress short of process area or maturity level
• to allay concerns and build support among stakeholders
China's Quality Seminar 45
n The staged representation can provide structure for a continuous-focus initiative:• as a guide for big picture “strategic” planning• to “chunk” higher-granularity activities for senior management
• as a means for representing high-level success in “industry standard” terms key stakeholders will more readily understand
CMMI Can Benefit Youn CMMI provides
q Efficient, effective assessment and improvement across multiple process disciplines in an organization
q Improvements to best practices incorporated from
China's Quality Seminar 46
q Improvements to best practices incorporated from the Software CMM
q A common, integrated vision of improvement for all elements of an organization
q A means of representing new discipline-specific information in a standard, proven process-improvement context
Conclusions
n Developing Quality Software is essential in any software enterprise
n Software Testing is a close companion of Software Quality
n Quality is essential in Delivering Offshore IT Servicesn Quality Impacts Business Results
China's Quality Seminar 47
n Quality Impacts Business Resultsn Quality Certifications are highly recommendedn The SEI vision is to develop software reducing the testing effort
n The importance of “Process” in Software Developmentn CMMI can benefit any Software enterprise