Peer Review Process

Formal and Informal Peer Reviews

2

Objectives

Understand the Peer Review process Subprocess to the Verification and Validation

Process Understand how defect measures are used

for process improvement

3

Peer Review Because...

Deliver quality products Catch defects earlier in the lifecycle Improve skills and knowledge Improve process performance

4

CMMI Basis

Peer reviews are part of Verification “...to ensure that selected work products meet

their specified requirements.” SG 2: “Peer reviews are performed on

selected work products.” The Peer Review Process is an appendix to

the Verification and Validation Process BPS/16-Verification and Validation

5

Peer Review Roles

Employee Responsibilities

Project Manager 1. Identifies and schedules reviews2. Identifies personnel resource needs3. Assigns moderator4. Does not attend the review

Author 1. Creates work product to be reviewed2. Prepares and present overview, as needed3. Completes rework

Moderator 1. Works with producer/developer/author to schedule formal peer review2. Ensures that review discussion focuses on the product and not the

producer/developer/author.3. Briefs inspection team roles4. Distributes inspection materials5. Ensures that the recorder documents all errors and action items

Quality Assurance (QA) 1. Ensures PR process is followed

Reviewer 1. Reviews work products for errors and issues, recording review time spent and findings on an inspection problem report (IPR)

Scribe (recorder) 1. Completes the IPR, recording duration of the PR and all defects and issues identified

6

Terms

Defect—A flaw that causes a work product to fail to perform a required function or behave contrary to documented expectations

Major defect—A defect that if not corrected prevents the product from meeting its requirements. Requires correction before the work product may be promoted to the next phase in the lifecycle.

Minor defect—A defect that if not corrected means the product will not meet specifications, design, or standards completely, but will not prevent the product from otherwise operating or serving its purpose.

A project manager may decide to permit a work product to be promoted to the next phase in the lifecycle without all minor defects being corrected. This decision must be documented on the IPR form.

7

Terms

Other defect—A deviation from standards and specifications; or a trivial issue, such as a format/spelling/grammar errors or other typos that do not affect meaning.

A large number of format/spelling/grammar errors or other typos may constitute one major defect because a document that does not get the spelling/grammar/formatting right may be perceived as unlikely to have the facts right either.

If disregard for coding standards results in code that, while meeting the requirements, will be difficult to maintained or will complicate current or future integration or interfaces, it may constitute a major defect.

Defect type—A classification of a defect indicating what sort of error was committed. The choice of types differs depending on what is being reviewed.

8

Material Reviewed Code Design Document Requirements

Type of Review Formal Informal

Types of Peer Reviews

9

Material Reviewed

This table shows a standard arrangement of work products and types of reviews, but this scheme may be modified and documented in the SDP.

Work Product PR Type Recommended Participants

Major software units Required Formal Experts in the coding language; testers

Major software unit design Required Informal Requirements analysts, developers/coders; testers

Minor software units, updates Recommended Informal Another developer

Requirements Required Formal Requirements specialists, designers, coders, testers

System test description Required Formal Testers, technical writers, QA analysts

User's guide/release notes Recommended Informal Developers; requirements analysts; technical writers

10

Resources (in BPS)

Process /13-Verification and Validation

Peer Review Form

/13-Verification and Validation/Tools Guides and Templates/Peer Review Form.xls

Guidelines /13-Verification and Validation/Tools Guides and Templates/Guidelines for <x> Reviews.doc

Standards /Organizational Process Assets/Standards

11

Formal Peer Reviews

Planning and preparing Conducting the peer review Documenting the findings Follow-up work

12

Formal PR: Plan and Prepare

Project manager selects reviewers Moderator sets time and place

Two days (at least) before the review, sends: Invitation Material to be reviewed IPR form Review guidelines Standard to be used

13

Formal PR: Conduct the Review

Discuss the work product, not the author Come prepared with a list of defects and

questions Set a time limit Do not solve problems during the peer

review, document them Consensus on major/minor/other, category

14

Formal PR: Document Findings

Select Type

Complete information

Size and time are required for measurements

15

Formal PR: Document Findings

Types listed on page 1 and in process

Disposition: Deferred, Closed, No Action

Defect Recording

Follow-Up Work

16

Formal PR: Document Findings

17

Formal PR: Follow-Up

Project manager and author determine what to correct

Finish IPR form and put it in the project’s SDLC Info/Peer Review directory.

QA checks to see whether action items are completed

Billing: Follow-Up Work on IPR form does not include defect

correction, just work to complete the form PR review, meeting, and form work bill to Peer

Reviews Defect correction work bill to Debug

18

Informal PR

Project manager assigns a reviewer Findings can be in IPR, e-mail, annotation, etc.

Findings still go in /Peer Reviews Author updates work products based on the review

results, consulting with the reviewer as needed. As in formal peer review follow-ups, the project

manager resolves disagreements over defects and other issues.

Once all the changes have been agreed to and made, the author forwards the modified work product to the project manager or to the CM group for inclusion in the project baseline.

19

Measurements

Collected in MA spreadsheet

Reviews Page

Dashboard Page

20

Additional Analysis

Def/Rev Hr. vs. Review %

-1

0

1

2

3

4

5

6

7

8

9

0% 10% 20% 30% 40% 50% 60% 70%

Review Time %

Def

/Rev

. Hou

r

Code Design Requirements Linear (Code) Linear (Design)

Negative correlation between % of IPR time spent on review and defects per review hour

21

Additional Analysis

Code Reviews: Review Minutes/LOC vs. Def/LOC

0.0

0.5

1.0

1.5

2.0

2.5

0.00 0.01 0.02 0.03 0.04 0.05 0.06

Def./LOC

Revi

ew M

inut

e/LO

C

Correlation between review time per LOC and defects found per LOC

22

Review

VER SP 2.1: Prepare for peer reviews How do we prepare?

Schedule Select work product for review Invitation Guidelines/standards Review material Training

23

Review

VER SP 2.2: Conduct peer reviews Things to remember during a peer review

Review the work, not the author Don’t solve defects, document them Develop consensus on defects, type, category List defects on IPR form Correct defects, update form

24

Review

VER SP 2.3: Analyze peer review data Data collected where? Used for what?

IPR forms MA spreadsheet Additional analysis Used for process improvement, to check process

efficiency, etc.