Lecture11 - Software Quality

7/23/2019 Lecture11 - Software Quality

1/26

28 May 2015 CSS 360 Spring 2015 Lecture 11

Conways Law:The structure of a computer

program reflects the structure ofthe organization that built it.


2/26

Outline Software quality what it is and who wants it? [discussed previously] Good enough quality

How can we measure quality?

Quality / Testing myths How, when, and what to test?

Bugs: stages and tracking over their lifetime

Test-Driven Development (TDD) with Unit tests

How can we improve software quality?

How to avert quality disasters?



3/26

References Professional Software Development, by Steve

McConnell

lots of discussions with professionals in the fieldover the years



4/26

Food for Thought:Quality in Different Contexts

The software Gold Rush fever periods Goal: being first-to-market in an unclaimed segment Customer base: tech savvy, willing to forgive bugs Project type: high-risk, potentially high pay-off Typical environment: two friends in a garage Processes: often code-and-fix, very informal

The in-between (post-Gold Rush) periods Goal: sustained, productive competition with others

Customer base: demands reliability Project type: lower-risk; lower, more predictable pay-off Typical environment: larger teams, formal processes Processes: careful, quality-driven development, emphasis

on quality (reliability, interoperability, usability, etc.)



5/26

Components of Quality

Quality comprises (but is not limited to): Requirements quality Design quality

Code quality Test quality Documentation quality

These are all aspects that QA engineers work on,

during the various software lifecycle stages.

Q:Given limited resources, which of these do youconsider more important to pay attention to? Why?



6/26


7/26

How Would You Test This?


Activity


8/26

The Quality / Testing Mindset Value proposition of quality assurance and testing: providing

information to the project team on the quality of the productand associated processes Notto break the code, but to discover where the code is already

broken, and to appropriately communicate these findings The conceptual model and assumptions about quality often

differ between testers and developers (non-testers) Broken/suspicious until proven otherwise vs.

Works/clean until proven otherwise Different mindsets of testers and developers

The two groups have somewhat different incentives. looking at a project through a different set of backgrounds/skills and thus could discover complementary useful aspects



9/26

Quality Assurance (QA) vs.Quality Control (QC) Software engineers and managers often treat QA

and QC as synonyms incorrectly! QC is reactive, after-the-fact, pure product testing. QA isproactive, provides informationto stakeholders

about the (objective) state of the project, the risks, andhow they may be mitigated/improved

QA looks holisticallyat the developmentprocess, and Proposes changes to improve it, so problems don't happen or recur

When problems happen, helps to discover them as close to thesource as possible (i.e., quickly and with lower project costs)

Overall goal of a QA organization is to: Prevent bugs as much as possible (through good processes) For bugs that are there, to discover them as early as possible (in

order to reduce costs and reduce waste)28 May 2015 CSS 360 Spring 2015 Lecture 11


10/26

Quality Assurance / Testing Myths

Myth 1: Developers dont own the code once itsreleased to testing.

Testers can help to ascertain the actual product quality,but developers must continue to be engaged for bug

fixes, critical decisions, code/design extensions, etc.

Myth 2: The QA organization is the bottleneckand is responsible for projects being late.

after all, arent they the last to touch the project, andto take precious project time while working on it.

Suggestion: Reviewing the value-stream mapcan helpto identify where problems originate.



11/26

Myth 3: Automation reduces the need for othertesting and QA.

Many things cant be automated (easily).

New features come up all the time, so someone needs towrite new tests.

Automated tests are only as good as the conceptualmodel on which they were based. The same challenge as developing a proxy: how much it can

understand vs. how simple it is to develop

Automation can yield both false positives and falsenegatives.


Quality Assurance / Testing Myths

(cont.)


12/26

Testing Approachesthroughout the Lifecycle Requirements / design / documentation inspections Acceptance / End-to-end testing System testing Integration testing Component testing Unit testing


V-modelillustrates mappingto stages in the lifecycle Question:Which correspond

to verification testing andwhich to validation testing?

http://thep

cweb.com/wp-content/up

loads/2008/08/v-model.jpg


13/26

How Do You Prioritize

What to Test? Factors to consider:

organizational mission statement

external requirements (e.g., safety, standards, etc.) feature priority

typical areas of high risk for bugs

recently developed or major features

new technologies external dependencies

complex algorithms



14/26

How Do You Know

When to Stop Testing? Consider the notion of good enough quality

Look at recent bug trends, including: If all high-priority test cases have passed

If there are no critical bugs left unresolved

If there have been no (very) recent code changes

(which may lead to problems if not testedthoroughly)

Hint: The above look like exit criteria.



15/26

Main Types of Testing

Black-box testingbased on specification (inputs)

White-box testingbased on dataflow (code paths)

Automated testing / Scripted testinguseful forrepeating tests

Manual testingexpensive, infrequent, error-prone,but sometimes necessary

Ad-hoc testing / Exploratory testinguseful for quickfeedback, usually when time is very limited

Regression testingbugs that were fixed must remainfixed (or be discovered quickly, ideally automatically)



16/26

Common Test Case Format Test case description/titlevery brief, intended for

quick identification among many other test caseswithout reading details

Specific testing instructionsdescribing how toperform the test, starting from a commonly knownplace/state

Expected outcome/resultsif the test succeeds, what

should be the results; this tells you how to identify atest case failure

Note: Including more aspects is possible, but optional.



17/26

A Bugs Life

Throughout its lifetime, a bug can transitionbetween several states:

Open (or re-opened)Work-in-progress

External Resolved

Cant reproduce By design

Wont fix Postpone Fixed

Closed



18/26

Tracking Bugs and Trends

Graph the number of bugs overtime, often grouped by variousproperties (e.g., open, howrecent, assigned to whom,priority, severity, etc.)

Track bug trends over the lifetimeof a project. Even if a bug isresolved, keep its record forproject tracking (and learning),and also in case it resurfaces.



19/26

Test-Driven Developmentwith Unit Tests

1. Write a test first (!) for a given (yet unwritten) classmethodthis way, tests serve as a specification (i.e.,expressing the intent) for the class code (methods)

2. Write stubs for the relevant method(s) of a class just enough to make the test compile3. Fail the testthis crucial step ensures falsifiability.4. Make changes to the class methodsjust enough to

pass the test

5. Pass the test ensuring that step #4 was successful6. Refactor (i.e., make changes), as necessary7. Pass the test ensuring that refactoring didnt break code8. Return to step #1 to cover additional intents of the class28 May 2015 CSS 360 Spring 2015 Lecture 11


20/26

Unit TestingAn Example

Example using JUnit: import org.junit.*;(where the JUnit methods are)import org.junit.Test;

@Beforefor setting up a test suite (i.e., set of tests) @Testfor describing an individual test in a suite Assert.assertEquals(commentString,

intendedValue, actualValue)

[Discussing sample JUnit code shown separately]



21/26

Best Practices for Testing

Tests must fail the first timeyou run them. to ensure that they arefalsifiable

The ideal tests are independent of each other: e.g.,

with no side effects and no special order of execution. Tests that are simple to write and quick to execute

create an incentive to be written and executed often.

Seek equivalence classesfor the domain youre testing.

Use an established testing methodology(e.g., SFDPO)rather than making up tests at random.



22/26

Important Practices for

Good Quality Processes Perform testing in an environment separate

from the production (live, online) environment.

The QA organization must have a seat at thetable of strategic decision makers.

The hierarchy of the QA organization mustideally be independent and not answerable tothe Development organization.


Activity


23/26

Mechanisms for Raising

the Quality of SoftwareAssume you are brought in on an ongoing software

project plagued by poor quality. What one or twoapproaches (mechanisms) would you propose to helpraise the quality of the software in production?

Make assumptions as needed, to concretize the question.


Activity

-

-

h f h


24/26

Mechanisms for Raising the

Quality of Software: Some IdeasWhich mechanisms may help your project(s)?a. Involvement / frequent iterations with customers and other

stakeholdersb. Pair programmingc. Code reviews (not limited to code; reqs/design review, etc.)

d. External auditinge. Using automated tools (e.g., static analysis, code coverage, IDEs,

etc.) to help discover non-trivial properties that affect qualityf. Refactoringg. Code integration (if not already in place)

h. Testing: integration testing, regression testing, acceptance testing;automated testing; test-driven development (with unit testing)i. Component reuse

j. Team building activitiesk. Establish (or ensure) clear responsibilities within the teaml. Realistic up-to-date scheduling



25/26

Perennial Recipes for Disaster(a.k.a. Poor Quality Products)

Ignore what the customers say they want developers surely must know better.

Put in all the features that could potentially ever be

useful. Do not worry about quality (and ignore related

prudent practices) until a deadline looms or someonecomplains loudly.

Do not waste time on design or documentation afterall, code is the most important thing, and time isalready short to do all that needs to be done anyway.



26/26

Questions?


Lecture11 - Software Quality

Documents

Transcript of Lecture11 - Software Quality