C 1998 Open Software Associates Open Software Associates Software Engineering: Analysis and Design...

c 19

98 O

pen

Softw

are

Asso

ciate

s

Open Software Associates Open Software Associates

Software Engineering: Analysis and Design (CSE3308)

Lecture 11b: Software Development in Industry

CSE3308/CSC3080/DMS/2000/26

CSE3308: Software Engineering - Analysis and DesignCSE3308: Software Engineering - Analysis and DesignAn Industry PerspectiveAn Industry Perspective

c 19

98 O

pen

Softw

are

Asso

ciate

s

IntroductionIntroduction

Lecture 11b

Software Development Process Topics• Software Releases• Revision Control• Software Testing• Memory Usage Testing• GUI Testing


c 19

98 O

pen

Softw

are

Asso

ciate

s

Software ReleasesSoftware Releases

Software Releases

Release levels Tree of revisions Release Checklists Release Checklist Example Release Criteria Defect Severity Support Objectives


c 19

98 O

pen

Softw

are

Asso

ciate

s


Release levels

Four levels of controlled releases at OSA:• Engineering Build (build)

– result of a product/system build– regularly built from ‘leaf code’ in the trunk and/or CPE branch– tested with automated test suite

• Alpha Release (alpha)– shipped only to OSA internal staff– tested with automated test suite, and against Alpha Release Checklist

• Beta Release (beta)– shipped only to nominated customer(s) and their OSA support staff.– tested with automated test suite, and against Beta Release Checklist

• Manufacturing Release (MR)– shipped with new product and/or upgrade sales, and to all customers with a

support contract.– tested with automated test suite, and against MR Release Checklist


c 19

98 O

pen

Softw

are

Asso

ciate

s


Release levels

Variations of an MR level Release:• Patch

– a update to a Manufacturing Release to correct defects– built from code in a CPE (Customer Product Engineering) branch of the

‘revision tree’.– shipped with new product and/or upgrade sales, and to all customers with a

support contract– tested as per an MR, with additional testing to ensure that it can be applied

over a particular MR• General Availability Release (GA)

– aka: Gold Release– concurrent availability of MR level releases for a product on all of its supported

platforms.– tested as per an MR, for all supported platforms.– shipped with new product and/or upgrade sales, and to all customers with a

support contract


c 19

98 O

pen

Softw

are

Asso

ciate

s


Tree of revisions

trunk revisions are on central line branch revisions are to the sides trunk revisions, branches and branch revisions can be labeled

symbolically.

3.0 3.48 4.0 4.12 4.36 ...

3.48.1 3.48.1.1

4.12.1

4.36.1 4.36.1.7

trunk

d420p6d420p7

d410d350

CPE branch CPE branch

branch

rev.

d420

d350p1

d300 d400

4.12.1.1

Product: OpenUI DevelopmentFile: windows1.cxx


c 19

98 O

pen

Softw

are

Asso

ciate

s


Release Checklists

Checklists• used to guide the release process

– document the release procedure• confirm that all release criteria

have been met met• record notes from release

process• record sign-offs with dates• are accessible on-line through

CGI based forms

Checklists identify:• Product Name• Package Name and Build Number• Release Platform/s• Product Distribution Details• Product Changes & Defect Fixes

ExampleRelease Checklist Sections

• Release Procedure• Product Configuration• Distribution Details• Product Change Details• LPE Checklist• Product Development Checklist• Quality Checklist• Support Checklist• Manufacturing Checklist • Release Sign-Off• Post-Signoff Processing• Announcement Approval• Notes


c 19

98 O

pen

Softw

are

Asso

ciate

s


Release Checklist Example

Ref: Release Sign-Off Form


c 19

98 O

pen

Softw

are

Asso

ciate

s


Release Criteria

Test Coverage

Interfaces Parameters Defect Rate/MNCSSReleaseType Ext % Int % Ext % Int %

Depth(PFC)

Reliability(hours) Critical Serious Medium

EB 80 0 20 0 40 N/A N/A N/A N/A

alpha 100 0 50 0 40 24 0 100 500

beta 100 80 70 50 60 48 0 50 300

MR 100 100 100 100 85 120 0 50 100

Test Coverage, Reliability, & Defects The following test coverage, minimum reliability, and defect rate criteria will be applied. NB: These criteria apply to both customer reported and OSA reported defects.

• For new or modified code

Release Quality Improvement For MR releases, quality as measured against defect, reliability, and test coverage criteria will be on a par with, or better than, the prior release of the same product.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Defect Severity

CriticalThe product can't be used. No work-around exists.• Data corrupted or lost • System failure or lockup occurs • Incorrect answers when necessary for useful operation • Program aborts when used correctly • Corruption of O/S or system due to incorrect use of product

SeriousThe product can't be used but a work-around exists.• Much harder to use than reasonably expected • Defect with awkward or inefficient work-around • Defect with misleading output • Documentation functionally incorrect • Program aborts when used incorrectly


c 19

98 O

pen

Softw

are

Asso

ciate

s


Defect Severity

Medium / ModerateThe product is useable with moderate workaround that can be provided.• Confusing user interface • Program aborts in malicious use • Defect is not serious to the operations of the user • Inefficient algorithm leads to substantial waste of time/space • Defect causes a preferred feature not to be used

LowThe product is useable with a simple workaround or fix.• Grammatical error in output • Ugly screen layout • User misunderstanding because of somewhat unclear documentation • Cosmetic documentation error where correct use of the product is

obvious


c 19

98 O

pen

Softw

are

Asso

ciate

s


Support Objectives Verification and Grading Period for Defects

The Verification and Grading Period is the time from receipt of a Problem Report, until it is verified, analysed, and graded.

Rectification Period for DefectsThe Rectification Period includes the Verification and Grading Period. i.e. it is the total time from the first customer report of the defect, until a defect repair is delivered to the customer.

Level Verification and GradingPeriods

Rectification Period

Critical 2 hours 48 hours24 hours for Optus

Serious 24 hours4 hours for Optus

14 days2 days for Optus

Medium 48 hours Next MR

Low 48 hours Best efforts


c 19

98 O

pen

Softw

are

Asso

ciate

s

Revision ControlRevision Control

Revision Control

Revision Control Cron brings it all together Daily RCS logs Review processes Formal review guidelines


c 19

98 O

pen

Softw

are

Asso

ciate

s


Revision control

Functions of a revision control system:• Maintenance of a tree of revisions.

– RCS is used to maintain a complete history of changes (revisions) to a ‘source’ file.– In addition to the text of each revision, RCS stores the author, date and time of check in,

and a log message summarizing the change.– RCS can maintain separate lines of development for each file. It stores a tree structure that

represents the ancestral relationships among revisions.

• Resolution of access conflicts.– When two or more people try to modify the same revision of a file, RCS alerts them and

prevents one modification from corrupting the other.

• Merging of revisions and resolution of conflicts.– Two separate lines of development of a file can be coalesced by merging. If the revisions to

be merged affect the same lines of a file, RCS flags the overlapping changes.

• Release and configuration control.– Revisions can be assigned symbolic names or labels. Labels can be used to identify a file

revision in relation to Product Releases, and development milestones. When systematically applied, revision labeling can enable system configurations to be simply accessed and managed.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Cron brings it all together At OSA, every night, several machines do clean full build of the

product, picking up all changes made during the day. This requires a large set of recursive makefiles

Every code change must be reviewed by at least one other engineer. The name of the reviewer and the DTS (work unit) number that is fixed by the change must be included in the RCS log for the change.

Every night, a report is generated by collating the RCS logs of all changes made that day. This report is checked by a manager the following day.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Cron brings it all together When each work unit is resolved, the engineer must include a

labnotes file, describing the rationale behind the change, and a documentation file, if a documentation change is made.

Every night, a report is generated of all work units that have changed state, including the additional labnotes and documentation files. This report is checked by a manager the following day.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Daily RCS logs *** /src/forms/master/src/eqlib/common/fxform.cxx,v ***

revision 3.2date: 1996/04/12 04:25:39; author: shaun; state: Exp; lines: +4 -4DTS 9001. Previous change wasn't quite correct. Reviewers: pca, agw.===============================================*** /src/forms/master/src/eqlib/common/text1.cxx,v ***revision 3.33date: 1996/04/12 06:07:20; author: joeca; state: Exp; lines: +4 -3DTS9001: added member initialization for copy constructor to stop purifyUMR's. Rev michael===============================================*** /src/forms/master/src/eqlib/common/browser.cxx,v ***revision 1.11.3date: 1996/04/12 06:09:17; author: michael; state: Exp; lines: +38 -29DTS14306:Fixed msie i/f to work with dde calls. rev joeca.===============================================


c 19

98 O

pen

Softw

are

Asso

ciate

s


Review processes Reviews can be conducted at all stages in the development process: The purpose of holding a review includes:

• to increase familiarity of code and designs within projects• to clarify and understand interfaces• to gain exposure to examples of code / design (both good and bad)• to provide feedback to the author• to find problems• to reduce testing phases• to stabilize design before coding• to increase exposure to the review process itself

Reviews can be formal or informal OSA has a practice that all code changes are at least informally

reviewed.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Formal review guidelines In a formal review, each review attendee has a specific role to play:

• Author Organizes the review• Reader Conducts the review• Facilitator Controls the review• Inspector Attends the review to find problems• Observer Attends the review for experience

Reviews are intended to identify problems not to solve them Preparation by the Reader and Inspector is very important and may

take several hours. The reader "reads" code by describing what it is doing. The

Inspector identifies problem areas as the Reader reaches them. The Author responds to questions about problems areas. When there is agreement on the identification of a problem, the Facilitator notes the problem, and the review continues on.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Formal review guidelines Formal reviews can be quite intimidating, but well run they are an

excellent mechanism for improving the quality of the software and team effectiveness.


c 19

98 O

pen

Softw

are

Asso

ciate

s

Software TestingSoftware Testing

Software Testing

Testing approaches and strategies Software testing environment Regression suites Unit Test guidelines


c 19

98 O

pen

Softw

are

Asso

ciate

s


Testing approaches and strategies Their are two main approaches:

• white-box testing: done with an understanding of the internals of the implementation

• black-box testing: only concerned with the system’s external behavior

With black-box testing, two main strategies:• limits testing: push the limits of the system to see where and when it

fails• equivalence partitioning: functions and parameters within the

system can be divided into equivalence classes, one sample from each class is used for testing.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Regression suites A Regression Test Suite is used to identify regressed, or unexpected

charges in system behavior and performance, with respect to an established baseline.• A regression suite is a group of tests designed to thoroughly test all

required functionality of a system.• Typically, a regression suite is run against each new build of a product

to see if any of its functionality has regressed with respect to the established baseline.

Three important rules to follow when creating regression tests:• Build the capability to recover from errors into every logical component

of the regression• Structure the tests so that no component depends on the success of a

previous component• Start each component from a known base state


c 19

98 O

pen

Softw

are

Asso

ciate

s


Unit Test guidelines Some Guidelines:

• a programmer should avoid testing their own code• thoroughly inspect the results of each test• test cases must be written for the invalid and unexpected (as well as the

valid and expected)• make test cases and data repeatable• do not plan a testing effort on the tacit assumption that no errors will be

found• testing is an extremely creative and intellectually challenging task


c 19

98 O

pen

Softw

are

Asso

ciate

s


Software testing environment Some commercial software testing tools are particularly useful:

• Memory test tools, e.g. Purify and TestCenter• GUI Test Tools, e.g. QA Partner and X Runner/WinRunner

Memory Test Tools• Memory errors in languages such as C and C++ are easy to make and

can be quite difficult to find.• Memory Test Tools provide a form of internal limits testing, often

detecting problems which become critical in extreme situations. GUI Test Tools

• File-based programs can be tested by comparing actual output files with expected output files.

• For GUI programs, however, the "output" is to the screen. This output is not fixed but depends on screen size, resolution, and positioning.

• Commercial tools provide specialized GUI toolkit drivers which accommodate this flexibility.

c 19

98 O

pen

Softw

are

Asso

ciate

s

Open Software Associates Open Software Associates

end of lecture 11b


c 19

98 O

pen

Softw

are

Asso

ciate

s

Memory Usage TestingMemory Usage Testing

Memory Usage Testing

Memory usage testing Memory usage errors Memory usage error examples

• UMR: Uninitialised Memory Read• ABR/W: Array Bounds Read/Write• FMR/W: Freed Memory Read/Write• FUM: Freeing Unallocated Memory

How Purify Works Memory leaks


c 19

98 O

pen

Softw

are

Asso

ciate

s


Memory usage testing Recall, on UNIX, memory is available in the following areas:

• Data segment: initialized static memory• BSS segment: zeroed static memory• Stack: parameters and local variables• Heap: malloc'd memory


c 19

98 O

pen

Softw

are

Asso

ciate

s


Memory usage errors A number of memory errors can easily occur:

• UMR: Uninitialised memory read (stack, heap)• ABR/W: Array bounds read/write (heap)• FMR/W:Freed memory read/write (heap)• FUM: Freeing Unallocated Memory (heap)• NPR/W: Null Pointer Read/Write

Error detection• In C and C++, these errors are only detected when they cause a

segmentation violation, often long after a detectable error has occurred.• Other languages provide checks which may detect errors earlier, but for

debugging purposes the stack backtrace of who allocated the current or related chunks of memory is most useful

Code should not leave home without being Purified.


c 19

98 O

pen

Softw

are

Asso

ciate

s


UMR: Uninitialised Memory Read A UMR is a warning that uninitialised memory is about to be read.

For example:

• main(){ int num1, num2; num1 = num2 + 1; /* num2 is used */ exit (0)}

will yield the following report:

• UMR: Uninitialised memory read:* This is occurring while in: main [line 5, test.c]* Reading 4 bytes from 0xf7fff348 on stack.* This is local variable "num2" in function main.


c 19

98 O

pen

Softw

are

Asso

ciate

s


UMR: Uninitialised Memory Read Often, uninitialised memory will be zero, especially during unit

testing. Your program will seem to perform correctly but the UMR will eventually cause incorrect behavior.

If the memory about to be read has been allocated on the heap, then you get a stack backtrace of who allocated the memory:

• typedef struct{ char *data_p; int length;} string;

main(){ string *sptr; int slen;

sptr = (string *) malloc(sizeof(string)); slen = sptr->length + 1; exit(0);}


c 19

98 O

pen

Softw

are

Asso

ciate

s


UMR: Uninitialised Memory Read will yield the following report:

• UMR: Uninitialized memory read:* This is occurring while in: main [line 13, test.c]* Reading 4 bytes from 0x5f524 in the heap.* Address 0x5f524 is 4 bytes into a malloc'd block at 0x5f520 of 8bytes.* This block was allocated from: malloc [rtlib.o] main [line 12, test.c]


c 19

98 O

pen

Softw

are

Asso

ciate

s


ABR/W: Array Bounds Read/Write An ABR/W indicates that the program is about to read or write a

value before or after an allocated block. ABR indicates a warning, ABW indicates a corrupting error. For example:

• main(){ char *orig = "test"; char * copy;

copy = (char *) malloc(strlen(orig)); strcpy(copy, orig); free(copy); exit(0);}


c 19

98 O

pen

Softw

are

Asso

ciate

s


ABR/W: Array Bounds Read/Write will yield the following report:

• ABW: Array bounds write:* This is occurring while in: strcpy [rtlib.o] main [line 7, test.c]* Writing 5 bytes to 0x5f558 in the heap (1 byte at 0x5f55c illegal).* Address 0x5f558 is at the beginning of a malloc'd block of 4 bytes.* This block was allocated from: malloc [rtlib.o] main [line 6, test.c]


c 19

98 O

pen

Softw

are

Asso

ciate

s


FMR/W: Freed Memory Read/Write An FMR/W indicates that the program is about to read from or write

to heap memory that has already been freed. FMR indicates a warning, FMW indicates a corrupting error. For example:

• main(){ int * num_p; int other_num;

num_p = (int *) malloc(sizeof(int)); free(num_p); other_num = *num_p + 1; exit(0);}


c 19

98 O

pen

Softw

are

Asso

ciate

s


FMR/W: Freed Memory Read/Write yields the following report:

• FMR: Free memory read:* This is occurring while in: main [line 8, test.c]* Reading 4 bytes from 0x5f530 in the heap.* Address 0x5f530 is at the beginning of a freed block of 4bytes.* This block was allocated from: malloc [rtlib.o] main [line 6, test.c]* There have been 0 frees since this block was freed from: free [rtlib.o] main [line 7, test.c]


c 19

98 O

pen

Softw

are

Asso

ciate

s


FUM: Freeing Unallocated Memory An FUM indicates that the program is trying to free unallocated

memory (duplicate free or free of bad heap pointer). FUM indicates a warning. For example:

• main(){ char *mem_p;

mem_p = (char *) malloc(10); free(mem_p); free(mem_p); exit(0);}


c 19

98 O

pen

Softw

are

Asso

ciate

s


FUM: Freeing Unallocated Memory yields the following report:

• FUM: Freeing unallocated memory:* This is occurring while in: free [rtlib.o] main [line 7, test.c]* Attempting to free block at 0x5f520 already freed.* This block was allocated from: malloc [rtlib.o] main [line 5, test.c]* There have been 1 frees since this block was freed from: free [rtlib.o] main [line 6, test.c]

This error often occurs due to confusion about pointer ownership.


c 19

98 O

pen

Softw

are

Asso

ciate

s


How Purify Works Purify checks every read, write, allocation and free of memory, and

verifies whether these operations have been done legally. Conceptually, Purify color codes the memory. It is coded into three

colors at the byte level:• Red: unallocated and uninitialised memory. It cannot be read, written

or freed. The heap is initially red.• Yellow: allocated but not initialised. It can be written or freed, but

cannot be read. Memory returned by malloc, and stack memory on function entry are set yellow.

• Green: allocated and initialised. It can be read written or freed. The data and BSS segments are automatically set to green.

Purify also maintains stack backtraces to specific depth for all memory operations.


c 19

98 O

pen

Softw

are

Asso

ciate

s


How Purify Works At link time, Purify inserts checking code into the program to

maintain the color-coded map of memory and detect errors. Since it works at the object code level, it is able to detect errors in

the entire program, including files for which you do not have the source.


c 19

98 O

pen

Softw

are

Asso

ciate

s


Memory leaks Memory leaks are memory chunks that cannot be accessed or

freed. They slow down a program by fragmenting or using memory, causing increased paging, or worse causing a program to run out of memory.

Purify gives a memory leakage report at the end of each run, indicating how much memory was leaked from each distinct memory allocation stack backtrace.


c 19

98 O

pen

Softw

are

Asso

ciate

s

GUI TestingGUI Testing

GUI Testing Graphical user interface (GUI) applications consist of a set of

windows and graphical controls, such as buttons, menus, edit texts lists, and combo-boxes

GUI testing presents some special difficulties:• System input/output is not fixed but depends on fonts, colors, screen

size, screen resolution and positioning - record/playback is very sensitive to these changes

• Screen snapshots generate too much low level data to be useful as expected output - one small change may invalidate all expected output

• Tests may need to be portable across Microsoft Windows, OSF/Motif, Apple Macintosh, OS/2 Presentation Manager

These problems are overcome with special GUI testing tools such as QA Partner and WinRunner.


c 19

98 O

pen

Softw

are

Asso

ciate

s


How QA Partner works

The QAP Recorder gets all the information the GUI has about each window and control on the screen, so the test scripts can refer to them by name, not by location and appearance.

Test scripts are written in 4Test, a C++ like language

Each test script has the general form:• initialize GUI• perform test• check result• handle exceptions

QA Partner(+ script)

QAPRecorder

GUI Program

QAPAgent


c 19

98 O

pen

Softw

are

Asso

ciate

s



The QAP Recorder gets all the information the GUI has about each window and control on the screen, so the test scripts can refer to them by name, not by location and appearance.

Test scripts are written in 4Test, a C++ like language

Each test script has the general form:• initialize GUI• perform test• check result• handle exceptions


QAPRecorder

GUI Program


c 19

98 O

pen

Softw

are

Asso

ciate

s



QA Partner runs the test script to perform test cases.• The QAP Agent executes GUI actions

and queries against the program under test.

• Many Agents can simultaneously execute a single QAP script to enable distributed or multi-host testing.

To allow a suite of Test Cases to be run, QA Partner implements an on-error recovery system that can restore the application back to a pre-defined state.


QAPRecorder

GUI Program

QAPAgent


c 19

98 O

pen

Softw

are

Asso

ciate

s


Referring to GUI Objects Windows and controls in a GUI are organized as a hierarchy

• Each window and control is an instance of a class• All windows or controls have associated text, such as the title of a window,

the label of a button, the label of a menu GUI objects are identified by a sequence of components corresponding

to the object hierarchy.• Components can be separated by ”."• Each component can consist of:

– the object's class, denoted by "[class]"– the object's name can be specified as follows:

• its associated text• internal GUI ID• ordinal number in relation to other objects at that level in hierarchy,• prior static text associated with object

• For example:– OurApp.Open.Filesis a listbox on a dialog that is

identified by “Files”, “#1”, “$4046”,

“(56, 94)”– OurApp.Open.OK is a button on a dialog


c 19

98 O

pen

Softw

are

Asso

ciate

s


Properties and operations on objects Each class has a built-in set of attributes and methods For example the built-in window class Menu includes these

attributes:• lsContents The contents of the menu• bEnabled Whether the menu is enabled

It also includes these methods:• GetContents Gets the contents of the menu

(an enquiry method)• IsEnabled Checks if the menu is enabled

(a state query method)• Pick Selects an item from the menu

(a GUI action method)• VerifyContents Verifies that the menu has the expected

contents(a verification method)


c 19

98 O

pen

Softw

are

Asso

ciate

s

Questions

C 1998 Open Software Associates Open Software Associates Software Engineering: Analysis and Design...

Documents

Transcript of C 1998 Open Software Associates Open Software Associates Software Engineering: Analysis and Design...