21 1

Continuing Best PracticesContinuing Best Practices

►Slide information taken in large part Slide information taken in large part from former Rational Corporation from former Rational Corporation slides. Considerably modified and slides. Considerably modified and supplemented for classroom use.supplemented for classroom use.

21 2

Practice 5: Verify Software QualityPractice 5: Verify Software Quality

Control Changes

Develop Iteratively

Use Component

Architectures

Manage Requirements

Model Visually VerifyVerify

QualityQuality

Quality, as used within Rational Unified Process, is defined as “The characteristic of having demonstrated the achievement of producing a product which meets or exceeds agreed upon requirements as measured by an agreed upon measures and criteria And is produced by an agreed upon process.

If done this way, the process can be repeated and managed

In most organizations, testing accounts for 30-50% of development costs! Yet most people believe software is not adequately tested when delivered. Testing is difficult; complete testing is impossible; a good process and automated tools help!

21 3

Software problems are 100 to 1000 times more costly to find and repair after deployment

Development Deployment

Cost

Practice 5: Verify Software QualityPractice 5: Verify Software Quality

One of the ways to improve quality: Test early and continuously! Test functionality, reliability; performance; Test architectural decisions early.

21 4

Iterative Development Permits Iterative Development Permits ContinuousContinuous TestingTesting – ensuring higher Quality – ensuring higher Quality

T I M ETest Test Test

TC

DR

Iteration 1 Iteration 2 Iteration 3

TC

DR

TC

DR

Test Life

CycleAssess

PlanDesign

ImplementExecute

Assess!

PlanDesign

ImplementExecute

Assess!

PlanDesign

ImplementExecute

21 5

Verifying Quality – and Continuous Verifying Quality – and Continuous Testing (continued)Testing (continued)

► Rather than test one time, spread testing out continuously. Part of each iteration – BUT (see below) Part of each iteration – BUT (see below)

► Each iteration produces an executable release (not necessarily a product release…) Don’t think of an ‘executable’ as just an .exe or .dll. The Don’t think of an ‘executable’ as just an .exe or .dll. The

executables may be part of an architecture……executables may be part of an architecture…… Each iteration is tested and integrated into an evolving application.

► Note: Each ‘phase’ has one or more iterations, and each phase has milestones!

► Be careful: The ‘extent (how much)’ of R, D, C, T depends on which phase the iteration is in!

► (See your drawings of the RUP)See your drawings of the RUP)

21 6

More…recall – talking about Verifying More…recall – talking about Verifying QualityQuality

► Cannot ‘engineer in’ quality; Must be threaded throughout development!

► Notice: Continuous Testing and integration! DistributesDistributes testing….ensures higher quality testing….ensures higher quality A fundamental design concept: Divide and Conquer!A fundamental design concept: Divide and Conquer!

► At end, entire system tested as a whole.► Many errors can be found early; fixed while repair costs are

inexpensive► Architectural decisions (key decisions) tested early avoiding

disastrous problems later. ► These features greatly reduce risks and liability of delivering

poor quality systems.

21 7

Testing in an Iterative EnvironmentTesting in an Iterative Environment R

eq

uir

em

en

tsR

eq

uir

em

en

ts

Test Suite 1Test Suite 1

Iteration 2Iteration 2 Iteration 3Iteration 3 Iteration 4Iteration 4

Test Suite 2Test Suite 2 Test Suite 3Test Suite 3 Test Suite 4Test Suite 4

Iteration 1Iteration 1

Au

tom

ate

d T

est

sA

uto

mate

d T

est

s

Continuous integration!!! (one of the major problems of SDLC!)We will produce automated tests (IF AVAILABLE in your ‘environment’)As new requirements are added in iterations, new tests are generated and run.This means that some tests will be rerun – part of ‘Regression Testing.’

21 8

Quality – What is it?Quality – What is it?

► It is an elusive termIt is an elusive term►Means different things to different Means different things to different

peoplepeople

Next few slides from OOSE text slides; modified

21 9

Software Quality...Software Quality...►Usability - Users can learn it fast and get theirUsability - Users can learn it fast and get their

job done easily job done easily (usually addressed in Interface)(usually addressed in Interface)

►Efficiency - It doesn’t waste resources like CPUEfficiency - It doesn’t waste resources like CPU time and memory time and memory (addressed in execution)(addressed in execution)

►Reliability - It does what it is required to doReliability - It does what it is required to do

without failing (MTBF, MTTR…)without failing (MTBF, MTTR…)►Maintainability - It can be easily changedMaintainability - It can be easily changed►Reusability - Its parts can be used in otherReusability - Its parts can be used in other

projects, so reprogramming is not neededprojects, so reprogramming is not needed

21 10

Software Quality...again, means Software Quality...again, means differentdifferent things to things to differentdifferent

stakeholdersstakeholders

QUALITY SOFTWARE

Developer: easy to design; easy to maintain; easy to reuse its parts

User: easy to learn; efficient to use; helps get work done

Customer: solves problems at an acceptable cost in terms of money paid and resources used

Development manager: sells more and pleases customers while costing less to develop and maintain

21 11

Software QualitySoftware Quality►The different qualities can The different qualities can conflictconflict.. Increasing efficiency can reduce maintainability or Increasing efficiency can reduce maintainability or reusabilityreusability

Increasing usability can reduce efficiencyIncreasing usability can reduce efficiency Increasing usability can reduce maintainabilityIncreasing usability can reduce maintainability Increasing maintainability can reduce efficiency, etc!Increasing maintainability can reduce efficiency, etc!

►Setting Setting objectives for qualityobjectives for quality is a key is a key engineering activityengineering activity Then Then designdesign to meet these objectives to meet these objectives Avoids ‘over-engineering’ which wastes moneyAvoids ‘over-engineering’ which wastes money Obtain the highest possible reliability using a fixed budgetObtain the highest possible reliability using a fixed budget

21 12

Short Term Vs. Long Term Short Term Vs. Long Term QualityQuality

►Short term:Short term: Does the software meet the customer’s Does the software meet the customer’s immediateimmediate

needs? needs? Is it sufficiently efficient for the volume of data we Is it sufficiently efficient for the volume of data we

have have todaytoday??

►Long term:Long term: MaintainabilityMaintainability Customer’s future needs Customer’s future needs ScalabilityScalability

21 13

Dimensions of Software QualityDimensions of Software Quality

Functionality

Reliability

Application Performance

System Performance

Does my app do what’s required?

Does my app leak memory?

Does my app respond acceptably?

Does my system perform under production load?

Test cases for each scenario implemented

Analysis tools and code instrumentation

Check performance for each use-case/scenario implemented

Test performance of all use-cases under authentic and worst-case load

Type Why? How?

For each iteration, do the ‘above’ software quality checks.Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications!

21 14

Problems Addressed by Verifying Problems Addressed by Verifying QualityQuality

Testing provides Testing provides objectiveobjective project status project status assessmentassessment

Objective assessment exposes Objective assessment exposes inconsistencies early inconsistencies early ((continuous integration helps!)continuous integration helps!)

Testing and verification are Testing and verification are focused on high risk areasfocused on high risk areas

Defects are found earlier and Defects are found earlier and are less expensive to fix are less expensive to fix (because ‘testing’ is (because ‘testing’ is distributed…distributed…

Automated testing tools Automated testing tools provide testing for reliability, provide testing for reliability, functionality, and performancefunctionality, and performance

Root CausesRoot Causes SolutionsSolutions Insufficient requirementsInsufficient requirements Ambiguous Ambiguous

communicationscommunications Brittle architecturesBrittle architectures Overwhelming complexityOverwhelming complexity Subjective assessment Subjective assessment Undetected Undetected

inconsistenciesinconsistencies

Poor testingPoor testing Waterfall developmentWaterfall development Uncontrolled changeUncontrolled change

Insufficient Insufficient automationautomation

21 15

Practice 6: Practice 6: ControlControl ChangesChanges to to SoftwareSoftware

Control ChangesControl Changes

Develop Iteratively

Use Component

Architectures

Manage Requirements

Model Visually

VerifyQuality

Must recognize that we CANNOT STOP CHANGE. Our goal is to Manage Change!The only ‘constant’ is ‘change!’Must control How and When control is introduced and who introduces the changes. This DOES NOT MEAN that we absolutely accept ALL changes, But…(Discuss!)Want a process that can respond to change…(RUP)Must synchronize Change across development teams and locations too. (What impacts do proposed changes have on our architecture!)

21 16

Without explicit control, parallel development degrades to chaos!!!!

Practice 6: Control Changes to Practice 6: Control Changes to SoftwareSoftware

► Consider: we often have:Consider: we often have: Multiple developersMultiple developers Multiple teamsMultiple teams Multiple sitesMultiple sites Multiple iterationsMultiple iterations Multiple releasesMultiple releases Multiple projectsMultiple projects Multiple platformsMultiple platforms

May have multiple developers organized into different teams at multiple sites all working together on multiple iterations, releases, products, and platforms (mostly based on the software architecture)

All at the same time!

21 19

Problems Addressed by Controlling Problems Addressed by Controlling ChangesChanges

Requirements change workflow is Requirements change workflow is defined and repeatabledefined and repeatable

Change requests facilitate clear Change requests facilitate clear communicationscommunications

Isolated workspaces reduce Isolated workspaces reduce interference from parallel workinterference from parallel work

Change rate statistics are good Change rate statistics are good metrics for objectively assessing metrics for objectively assessing project statusproject status

Workspaces contain all artifacts, Workspaces contain all artifacts, facilitating consistencyfacilitating consistency

Change propagation is Change propagation is controlledcontrolled

Changes maintained in a robust, Changes maintained in a robust, customizable systemcustomizable system

Root CausesRoot Causes SolutionsSolutions Insufficient Insufficient

requirementsrequirements Ambiguous Ambiguous

communicationscommunications Brittle architecturesBrittle architectures Overwhelming Overwhelming

complexitycomplexity Subjective assessment Subjective assessment Undetected Undetected

inconsistenciesinconsistencies Poor testingPoor testing Waterfall developmentWaterfall development Uncontrolled changeUncontrolled change Insufficient Insufficient

automationautomation

21 20

Best Practices Best Practices ReinforceReinforce Each Each OtherOther

Control Control ChangesChanges

Develop Develop IterativelyIteratively

Use Use ComponentComponent

ArchitecturesArchitectures

Model Model VisuallyVisually

VerifyVerifyQualityQuality

Ensures users involved as requirements evolve

Validates architectural decisions early on. Drives development,planning, change control. ….

Addresses complexity of design/implementation incrementally.Need tools/support environment!

Measures quality early and often.Continuous testing and integration

Evolves baselines incrementallyArchitecture teams localizing changes; Need CMS, Conf Control…

Manage Manage RequirementsRequirements

Remember: best practices yield best results WHEN USED COLLECTIVELY!

What does iterative development do??

21 21

Summary: Best Practices of Software Summary: Best Practices of Software EngineeringEngineering

The result is software that isThe result is software that is On TimeOn Time On Budget On Budget Meets Users NeedsMeets Users Needs

ProjectManager

PerformanceEngineer

ReleaseEngineer

Analyst

Developer

Tester

Control Control ChangesChanges

Develop IterativelyDevelop Iteratively

Use Use ComponentComponent

ArchitecturesArchitectures

Manage Manage RequirementsRequirements

Model Model VisuallyVisually VerifyVerify

QualityQuality