Active Design ReviewsActive Design Reviews

Doug PaidaDoug PaidaRoy MammenRoy Mammen

Sharan MudgalSharan MudgalJerry ChengJerry Cheng

OverviewOverview

What is design review?What is design review? Taxonomy Taxonomy Limitations of Conventional approachLimitations of Conventional approach Walking through the Active Review Walking through the Active Review

Process using an exampleProcess using an example ConclusionsConclusions

What is a design review?What is a design review?

A process of identifying faults in the A process of identifying faults in the design of a software programdesign of a software program

Review should uncover both errors Review should uncover both errors made in producing design made in producing design documents, and errors made earlier documents, and errors made earlier in the design processin the design process

TaxonomyTaxonomy

Structured Walkthrough author of the design and one or more

other developers familiar with the development activity are involved

Code Reading two or more developers read the code

independently Discuss with the author

TaxonomyTaxonomy Code Inspection

The process definition included planning, preparation , inspection, rework and follow up with specific roles define for the participants

author/reader paraphrases the material, while the participants ask question that may lead to the discovery of defects

Software Review The reviewers go through the material

individually using a checklist The author collects the checklists and

consolidates the results.

TaxonomyTaxonomy

Active Design Review The questionnaire focuses the reviewer's attention on a

particular set of issue reviewers concentrate on different sets of issues. The author meets with the reviewers individually to

review their findings.

Defect-Based Reading possible defects in requirements documents For each defect class a set of questions was developed

that would characterize the defect class The questions also characterize a set of steps that

should be performed while reading called scenarios.

the reader tries to answer the questions provided in the scenario.

TaxonomyTaxonomy Meetingless Reviews

As effective as meeting-based ones Individual strategies find more issues to resolve

but these have a higher false positive and duplication rates

The duplication rates can be reduced by specializing the roles during the process.

Computer-mediated Technical Reviews supporting clerical functions of distribution and

defect report collection and rework tracking. Aimed at synchronous distributed reviews where

the review is not required to be in the room.

Limitations of Conventional Limitations of Conventional approachapproach

Too much information to go through, Too much information to go through, and not enough time to do it and not enough time to do it thoroughlythoroughly

Unfamiliarity of individual reviewers Unfamiliarity of individual reviewers with the overall goals of the designwith the overall goals of the design

No single part of the design gets a No single part of the design gets a thorough and complete evaluationthorough and complete evaluation

(continued)(continued)

Burden is on reviewer to initiate actionBurden is on reviewer to initiate action

One-on-one interaction between One-on-one interaction between individual reviewers and design team is individual reviewers and design team is limitedlimited

No systematic procedure – generally No systematic procedure – generally more free-form type of formatmore free-form type of format

Active Design Review Active Design Review ProcessProcess Change from “general” review to a set Change from “general” review to a set

of more focused reviewsof more focused reviews

Use questionnaires to actively engage Use questionnaires to actively engage the reviewer in using the designthe reviewer in using the design

More opportunities for one-on-one More opportunities for one-on-one discussion between reviewer and discussion between reviewer and designerdesigner

A design exampleA design example

We have been asked to review a design We have been asked to review a design for a module which is part of an order for a module which is part of an order processing system for a hospital.processing system for a hospital.

The order processing system allows The order processing system allows users to order items for patients, such users to order items for patients, such as tests or medications.as tests or medications.

The module we are reviewing acts as an The module we are reviewing acts as an interface between the order database interface between the order database and other modules in the system, and other modules in the system, hiding details of the actual database hiding details of the actual database tables used.tables used.

Order Processing System

Add/Cancel Order Print/Display Orders

Database Interface

Database

Design Example (cont.)Design Example (cont.)

Orders

Order

Item

Module requirementsModule requirements

An order must contain at least these data An order must contain at least these data items:items: Patient IDPatient ID List of items being ordered. There must always be List of items being ordered. There must always be

at least one item in every order.at least one item in every order. Date/time the items are being ordered forDate/time the items are being ordered for Status of the order (active or cancelled)Status of the order (active or cancelled)

An item in an order contains these data items:An item in an order contains these data items: Item IDItem ID Quantity orderedQuantity ordered

Module requirementsModule requirements

Basic functionality requiredBasic functionality required Add a new order for a patientAdd a new order for a patient Change an order’s status to “cancelled”Change an order’s status to “cancelled” Obtain the number of orders a patient Obtain the number of orders a patient

hashas Iterate through all orders for a patientIterate through all orders for a patient Iterate through all items within an orderIterate through all items within an order

Module requirementsModule requirements

Actions that should be prohibitedActions that should be prohibited Making order active again once it is Making order active again once it is

cancelledcancelled Adding items to an order after it’s been Adding items to an order after it’s been

posted to the databaseposted to the database

Module specificationModule specification

Data in the Order classData in the Order class int patientIDint patientID int orderIDint orderID Date orderDateTimeDate orderDateTime int statusint status Vector itemsVector items

Module specificationModule specification

Access functions for Order classAccess functions for Order class void addItem(Item i)void addItem(Item i) Item getFirstItem()Item getFirstItem() Item getNextItem()Item getNextItem() int numItems()int numItems() void setStatusToCancelled()void setStatusToCancelled() Date getOrderDateTime()Date getOrderDateTime() int getStatus()int getStatus()

Module specificationModule specification

Data for Item classData for Item class int itemIDint itemID int quantityint quantity

Access functions for Item classAccess functions for Item class int getItemID()int getItemID() int getQuantity()int getQuantity()

Module specificationModule specification

Access functions for Orders classAccess functions for Orders class void submitOrder(Order o)void submitOrder(Order o) void updateOrder(Order o)void updateOrder(Order o) Order getOrder(int orderID)Order getOrder(int orderID) Order getFirstOrder(int patientID)Order getFirstOrder(int patientID) Order getNextOrder()Order getNextOrder() int numOrders(int patientID)int numOrders(int patientID)

Module specificationModule specification

Undesired EventsUndesired Events getNextOrder() called before getNextOrder() called before

getFirstOrder()getFirstOrder() addItem() called on an order that’s addItem() called on an order that’s

already been posted to the databasealready been posted to the database

Steps for Active Design Steps for Active Design ReviewReview

Prepare design and documentation Prepare design and documentation for reviewfor review

Identify for specialized reviewsIdentify for specialized reviews Identify the reviewers neededIdentify the reviewers needed Design the questionnairesDesign the questionnaires Conduct the reviewConduct the review

Active Design Review Active Design Review ProcessProcessStep 1: Prepare the design for reviewStep 1: Prepare the design for review Think about what criteria reviewers will use:Think about what criteria reviewers will use:

Well-structuredWell-structured SimpleSimple EfficientEfficient AdequateAdequate FlexibleFlexible PracticalPractical ImplementableImplementable StandardizedStandardized

Active Design Review Active Design Review ProcessProcessStep 1: Prepare the documentation for Step 1: Prepare the documentation for reviewreview Make assumptions explicitMake assumptions explicit

Module can record the order pertaining to a Module can record the order pertaining to a patient.patient.

It is possible to obtain all the orders for a It is possible to obtain all the orders for a patient.patient.

Module can determine & change the status of Module can determine & change the status of an order.an order.

The order always contains at least one itemThe order always contains at least one item The status of an order is always in one of the The status of an order is always in one of the

two states i.e active or cancelled.two states i.e active or cancelled.

Active Design Review Active Design Review ProcessProcessStep 1: Prepare the documentation for Step 1: Prepare the documentation for reviewreview Incorrect Usage Assumptions (UE)Incorrect Usage Assumptions (UE)

Cannot add or remove items once the Cannot add or remove items once the order is placed.order is placed.

Once an order is cancelled, the status Once an order is cancelled, the status cannot be set to active again.cannot be set to active again.

An item is always added with respect to An item is always added with respect to an order.an order.

Active Design Review Active Design Review ProcessProcessStep 2: Identify the specialized reviewsStep 2: Identify the specialized reviews Focus the reviewer’s attention on specific Focus the reviewer’s attention on specific

properties of the designproperties of the design Data Access Sufficiency.Data Access Sufficiency.

Provides all data required by the other modules ?Provides all data required by the other modules ? Assumption Sufficiency.Assumption Sufficiency.

Contains all assumptions needed by the user program to Contains all assumptions needed by the user program to efficiently access the DB ?efficiently access the DB ? Consistency between Assumptions and Consistency between Assumptions and

Functions.Functions.Assumptions? Access Functions? UE?Assumptions? Access Functions? UE?

Active Design Review Active Design Review ProcessProcessStep 3: Identify the reviewers neededStep 3: Identify the reviewers needed People with different perspectives People with different perspectives

and expertise are needed as and expertise are needed as reviewersreviewers Programmers and analysts who worked Programmers and analysts who worked

on the other modules in this order on the other modules in this order processing system. processing system.

Programmers and analysts familiar with Programmers and analysts familiar with hospital information systems in general.hospital information systems in general.

Active Design Review Active Design Review ProcessProcessStep 4: Design the questionnairesStep 4: Design the questionnaires Make reviewers take an active roleMake reviewers take an active role Make them use the documentationMake them use the documentation Phrase questions in an active wayPhrase questions in an active way

““Write down the exceptions that can Write down the exceptions that can occur” rather than “Are exceptions occur” rather than “Are exceptions defined for every program?”defined for every program?”

Active Design Review Active Design Review ProcessProcessStep 5: Conduct the reviewStep 5: Conduct the review Present an overview of the modulePresent an overview of the module Assign reviews to reviewersAssign reviews to reviewers Reviewers complete their reviews, meeting Reviewers complete their reviews, meeting

with designers as neededwith designers as needed Designers review completed Designers review completed

questionnaires, and meet with reviewers to questionnaires, and meet with reviewers to resolve questionsresolve questions

Designers produce new version of the Designers produce new version of the documentationdocumentation

Sample Review: Data Access Sample Review: Data Access SufficiencySufficiency

the access functions should be evaluated to the access functions should be evaluated to ensure that access is provided to all data required ensure that access is provided to all data required by the other modules by the other modules

piece of data about an order that cannot be piece of data about an order that cannot be retrieved through an access function but needed retrieved through an access function but needed reflects error in design.reflects error in design.

possible to modify the value of any piece of data possible to modify the value of any piece of data in violation of the stated requirements, then there in violation of the stated requirements, then there is a design error.is a design error.

The module specifications should be reviewed by The module specifications should be reviewed by programmers of the other order processing programmers of the other order processing system modules for their ability to satisfy these system modules for their ability to satisfy these criteria. criteria.

Example QuestionnaireExample QuestionnaireThe following questions should be used to The following questions should be used to

aid in this reviewaid in this review Using the access functions provided in the Using the access functions provided in the

module specification, write the code you would module specification, write the code you would use to obtain the date/time, item number and use to obtain the date/time, item number and quantity ordered for all active orders for a quantity ordered for all active orders for a patient.patient.

Attempt to come up with a sequence of access Attempt to come up with a sequence of access function calls that would allow a program to set function calls that would allow a program to set a cancelled order’s status to active.a cancelled order’s status to active.

For each access function provided, write down For each access function provided, write down the specific requirements from the the specific requirements from the requirements list that you believe the function requirements list that you believe the function was designed to meet. Make note of any was designed to meet. Make note of any functions that do not appear to satisfy any functions that do not appear to satisfy any specific requirements specific requirements

ConclusionsConclusions

Reviewers focus on those areas they are Reviewers focus on those areas they are best suited to evaluatebest suited to evaluate Time is used more wisely for all participantsTime is used more wisely for all participants More errors are likely to be foundMore errors are likely to be found

One-on-one communication with designers One-on-one communication with designers makes it easier for people to speak upmakes it easier for people to speak up

Few errors found not necessarily indicate Few errors found not necessarily indicate that the design is good, but that the that the design is good, but that the review process was not as effective as it review process was not as effective as it could have beencould have been