SE-280 Dr. Mark L. Hornick 1 Design and Code Reviews Review Checklists.

SE-280Dr. Mark L. Hornick

1

Design and Code ReviewsReview Checklists


2

Compared to reviews, defect removal in later phases is much more expensive.

Average defect fix time (minutes) [Xerox TSP team]

5

22

2

25

32

1405

1 10 100 1000 10000

Design rev iew

Design insp.

Code rev iew

Code insp.

Unit test

System test


3

Design and code reviews are used to find and fix defects early in the development process.

Inspections

Walkthroughs

Reviews

Appraisal methods

0

1

2

3

4

5

6

7

Design rev iew Code Rev iew Unit Test

Defects removed per hour (SEI data)

While unit test rarely has a yield above 50% (industry data, not PSP), experienced reviewers often achieve a yield of 70%.

In the PSP, we use only personal reviews; the TSP incorporates

team inspections.

Reviews generally remove defects more efficiently than testing.


4

PSP data from SEI and MSOE students confirm the efficiency of code reviews.

Defect Removal Rates (MSOE)

0.0

5.0

10.0

15.0

20.0

25.0

1A 2A 3A 4A 5A 6A 7A

Program

Def

ects

/Ho

ur

Compile

CR

Test

Defect Removal Rates (SEI)

0.0

5.0

10.0

15.0

20.0

25.0

30.0

1A 2A 3A 4A 5A 6A 7A 8A 9A 10A

Program

Def

ects

/Ho

ur

Compile

CR

Test

Note: "compile" values in MSOE data reflect C++, not Java.


5

To understand the advantage of reviews over testing, let's look at how each handles a defect.

Review Test

Defect existence

Understand context

Identify cause & how to fix

Implement fix time

See it Get unexpected result

Known Search

Already known Debug, isolate

Fast Slow


6

Appraisal/Failure Ratio

Data from previous SE-280

A/F Ratio

0

1

1

2

2

3

3

4

4

5

1A 2A 3A 4A 5A 6A 7A

A/F

Rat

io Average

Minimum

Maximum


7

Defect Removal EfficiencyUnit test

2-4 defects/hourRarely more than 50% found

Code reviews10 defects/hourExperienced reviewers - 70% found

2-5 times as many defects/hour as unit test!


8

Let's try calculating system test time, based

on different assumptions for review yield. Assume that 20 defects are injected in design,

and 30 in coding, for a total of 50 defects.Developmentphase 1

Developmentphase 2

Developmentphase 3

Inj Yld Def20 20

DLDR 70% 630 36

CR 70% 10.80 10.8

UT 50% 5.40 5.4

ST hours 126weeks 3

Yld Def20

50% 1040

50% 2020

50% 1010

2346

Yld Def20

10% 1848

10% 43.243.2

50% 21.621.6506

13


9

PSP reviews make use of Checklists, which are based on historical defect data.

• Past mistakes -> present mistakes

• Structures the process


10

public class Data {

Once you have a checklist and the product to be reviewed, how should you proceed?

public class Gui {

public class Calc {

Method arguments

Initialization

Loop conditions

Library usage

Checklist format?

The format of a checklist reflects the review process approach.

Checklist item "Calc" "GUI" "Data"

1 Copy/paste errors

2 Field/var/param initialization

3 Check params for method preconditions

4 All possible exceptions handled?

5 Check for possible null object references

In each cell: Enter # of defects found, zero, or "N/A"

How do we manage the quality of our reviews?

What can we do earlier in the process (Planning, DLDR, CR)?

In postmortem, we can calculate the review yields, but then it is too late to

do anything about them!

Planning

Design

DLDR

Coding

CR

Compile

UT

PM

Review rateGeneral guideline: < 200 LOC/hour

When reviewing designs or code, some "minor" details can make a big difference.

"Paper or plastic?"

Take a break!

Reviewing a printed listing, away from your computer, often makes defects more visible.

Reviews are more effective if they don’t immediately follow

creation of the artifact being reviewed.


14

Why Checklists again?

They help execute precise tasks Difficult to keep track of doing many

separate things A Review Checklist:

Defines review steps, in order Stages review progress Can be customized to your own

defect experience

Personal checklists often combine your personal defect "favorites" with some "standard" categories.

Your midterm report defect analysis should suggest some defect

categories worth looking out for.

You may also get some good ideas from the sample checklists in the textbook

(pages 175 and 184).

Make your checklist complete enough to support a thorough review.


16

Next steps

Analyze your own Quality Data – Midterm report PD already calculates:

Test defects/KLOC Total defects/KLOC Yield Appraisal COQ (%) Failure COQ (%) COQ A/F ratio

Action Modify your personal process

Analyze again - Final report) Results & Conclusions


17

Midterm Assignment reading

Review textbook pages 133-202. Download sample code review checklist. Review the midterm report assignment.

Carefully review requirements.

One part of the midterm report assignment is designing and executing a "report process".

Planning

??

??

??

??

??

??

PM

Process phases Size measure(s)?

Pages, sections?

Charts, tables?In Process Dashboard,

add/rename generic phases in the "Midterm Report" project.

Why are we doing this, anyway?

SE-280 Dr. Mark L. Hornick 1 Design and Code Reviews Review Checklists.

Documents

Transcript of SE-280 Dr. Mark L. Hornick 1 Design and Code Reviews Review Checklists.