Regression Testing of COTS

Component-Based Software System

A Thesis Submitted in partial fulfillment of the requirement for the Award

of the Degree of

Master of Technology IN

COMPUTER SCIENCE & ENGINEERING ((22000066--22000088))

Under the Supervision of Ms. Gagandeep

SSuubbmmiitttteedd BByy YYooggeesshh KKuummaarr

DDeeppaarrttmmeenntt ooff CCoommppuutteerr SScciieennccee PPuunnjjaabbii UUnniivveerrssiittyy,, PPaattiiaallaa..

1

Acknowledgment

While this report may bear only one name on the cover, it is the result of inputs from

others who have helped me during the thesis work. In an effort to recognize the

contributions of those who helped bring thesis to life, I would like to acknowledge and

thank them.

First and foremost, I sincerely thank my project supervisor, Ms. Gagandeep, for her

continuous guidance with the thesis work. Without her encouragement, this thesis would

not have been possible. I would also like to thank Dr. Gurpreet Singh Lehel, Head,

Department of Computer Science, for his help throughout.

I am also thankful to the Department of Computer Science, because I have utilized the

resources available here for research purposes, and this has enormously helped me

throughout the duration of this thesis.

And last but not least, I wish to express my eternal gratitude towards my family for their

everlasting support. I am greatly indebted to them.

Yogesh Kumar

2

Abstract

The primary role of component-based software engineering is to address the development

of systems as an assembly of parts (components), the development of parts as reusable

entities, and the maintenance and upgrading of systems by customizing and replacing

such parts. This requires established methodologies and tool support covering the entire

component and system lifecycle including technological, organizational, marketing, legal,

and other aspects. The traditional disciplines from software engineering need new

methodologies to support component -based development.

Component-based Software Engineering (CBSE) has become recognized as such a new

sub-discipline of Software Engineering. Component based software engineering (CBSE)

also offers convenience as well as fast design/ implementation. Testing UML design

models is challenging because a design model consists of many models that describe

different design views. An approach to generating inputs for testing UML designs

requires a mechanism that combines information from both the structural and behavioral

views of a design.

Regression testing is a testing process which is applied after a program is modified. It

involves testing the modified program with the old test cases in order to re-establish

confidence in the program. As a part of our research work, we have proposed a

methodology to generate regression test suite based on UML sequence diagrams.

We have constructed and implemented control flow graph (CFG) in C++ for the storage

of UML sequence diagrams. The sequence diagrams have been drawn with the help of

NET BEANS tool. We have practically experimented on different component from

different application domain. We have stored the sequence diagrams for each component

into the control flow graph (CFG) and traversed it for the generation of the regression test

suite.

The results have been evaluated by calculating the total running time for each component.

Our technique of automating the regression test suite generation in effective in terms of

reduction in number of test cases and time efficient as compared to manual technique.

3

.

TABLE OF CONTENTS

1. Introduction 1 1.1 Problem …………………………………………………………1 1.2 Overview of the solution and research challenges………………2 Scope of the research…………………..………………………...3 1.3 Thesis organization………………………………………………3-4 2. Regression Testing …………………………………………………5 2.1 Need for regression testing……………………………………….5 2.2 Types of regression testing……………………………………….6 2.3 Difference between testing and regression testing………………..7 2.4 Similarities between testing and regression testing……………....8 2.5 Problem of regression testing…………………………………….9 2.6 Retest approach…………………………………………………..9-13 3. Literature Survey………………………………………………….14 3.1 Techniques of regression testing………………………………..14-16 4. Case studies of the existing techniques …………………………. 18 4.1 Generating regression tests via model checking………………...18-26 4.1 Regression testing UML design…………………………………27-36 5. Related work in component-based Regression testing and UML.37-39 6. Control flow graph based criteria to generate regression test suite…………………………………………………………….40-46 7. Case Study …………………………………………………………..47 7.1 Case study with ATM using Net Beans Tools for UML sequence diagram…………………………………………………………....47-51

4

7.2 Case study with E-Ticket of Cinema for UML sequence diagram..52-56 7.3 Case study with Library Record System for UML sequence diagram……………………………………………………………57-61 8. Conclusion and Future work ………………………………………62 8.1 Evaluation of the case studies……………………………………62-64 8.2 Summary of the contribution………………………………………65 8.3 Future work…………………………………………………….......65 Reference……………………………………………………………….66-69

5

LIST OF TABLE

Table 4.1.1 Test suite of the window 98 notepad………………………19 Table 4.1.2 Test suite of window XP notepad………………………….20 Table 4.1.3 Test suite of comparator…………………………………...23 Table 4.1.4 Test suite of coverage selector ……………………………24-25 Table 4.1.5 Test suite of test generator………………………………...25-26 Table 4.2.1 Test suit T of the Program P (simple program of factorial)..30 Table 4.2.2 Test suit T’ of the Program P’ (modified program of factorial)..………………………………………………….32-33

Table 4.2.3 UML Test suits of combined of T and T’…………………33-34 Table 4.2.4 Final Test cases …………………………………………35-36 Table 7.1.1 Case scenarios based on the ATM sequence diagram of first version……………………………………………………...48

Table 7.1.2 Case scenarios based on the ATM sequence diagram of modified version …………………………………………...50

Table 7.1.3 Final Regression Test case generation from the ATM Component ………………………………………………….51

Table 7.2.1 Case scenarios of first version of the E-Ticket system of Cinema.. …………………………………………………..52-53

6

Table 7.2.2 Case scenarios of modified version of the E-Ticket system of Cinema ……………………………………………………54-55

Table 7.2.3 Final Regression Test case generation from the E-Ticket System of Cinema Component……………………………………..55-56

Table 7.3.1 Case scenarios of the first version of the Library Record System component……………………………………………58

Table 7.3.2 Case scenarios of the modified version of the Library Record System component …………………….………….59-60 Table 7.3.3 Final regression test suite for Library record system……...60-61

7

LIST OF FIGURES

Figure 2.1 Regression testing ……………………………………………6 Figure 2.2 The retest regression testing process for corrective regression testing………............................................................................11 Figure 2.3 An operational overview of Retest…………………................12 Figure 4.1.1 Functionalities of comparator…………….….........................22 Figure 4.1.2 Coverage Selector……………………………........................24 Figure 4.2.1 Object Method Directed Acyclic Graph (OMDAG) for simple program of factorial ...............................................29 Figure 4.2.2 Object Method Directed Acyclic Graph (OMDAG) for modified version of factorial………………………….....31 Figure 6.1 Working of the control flow graph (CFG) based criteria for generation of regression test suite…………………………......40 Figure 6.2 Sequence diagram of student enrollment system……………...41 Figure 7.1.1 First version of the ATM Sequence Diagram………………..47 Figure 7.1.2 Extending the functionality of first version of ATM sequence

Diagram…………………………………………………….49

Figure 7.2.1 Sequence diagram of first version of E-Ticket of Cinema component…………………………………………52 Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema……………………………………………………….54

8

Figure 7.3.1 Sequence diagram of the first version for the Library Record System………………………………………………………..57 Figure 7.3.2 Sequence diagram of the modified version for the Library Record system………………………………………………..59

Figure 8.1.1 Bar chart of the total number of regression test suite for ATM

Component…………………………………………………….62

Figure 8.1.2 Bar chart of the total number of regression test suite for E-

Ticket System for Cinema Component………………………63

Figure 8.1.3 Bar chart of the total number of regression test suite for Library

Record System Component…………………………………..64

9

Chapter 1

Introduction

1.1 Problem In recent years, Component-based software has been widely used in various application

domains and becomes a fairly popular software form. However, Due to the lack of

information about the externally developed components, system testers (i.e., component

users) generally can't perform effective testing (especially regression testing) on their

component based systems. Component users don't know the details about change in

component, so they aren't able to select the proper test cases to retest the modified system

[3].

Meanwhile, Component-Based Software Engineering (CBSE) is a continuously growing

field. However, there is a lack for good environment support for the CBSE activities,

mainly because most of the environments do not offer comprehensive, integrated support

for the full range of CBSE activities. The construction of this kind of environments

involves general environment issues, such as tool integration, and specific CBSE issues,

such as domain engineering and application composition. The goal of component-based

software engineering is to increase the productivity, quality [3].

In component-based software systems, regression test case selection is a challenge since

vendor of commercial-off-the-shelf (COTS) component does not release source code.

Regression testing is applied to modified software to provide confidence that modified

code behaves as intended, and does not adversely affect the behavior of unmodified code.

Regression testing plays an integral role in software maintenance; without proper

regression testing we are reluctant to release modified software [7].

One characteristic distinguishing regression testing from developmental testing is the

availability, at regression test time, of existing test suites. If we reuse such test suites to

retest a modified program, we can reduce the effort required to perform that testing.

Unfortunately, test suites can be large, and we may not have time to rerun all tests in such

suites. Thus, we must often restrict our efforts to a subset of the previously existing tests.

We call the problem of choosing an appropriate subset of an existing test suite the

10

selective retest problem; we call a method for solving this problem a selective retest

method [18].

We use the strategy for regression test selection in component based software

engineering. This is achieved by the component developer providing information about

the changes in the first one version to next and the component user utilizing this

information in regression test case selection. We use Unified Modeling Language (UML)

and its sequence diagram to formally specify the test case selection process. UML is the

accepted standard for component design, and by specifying the regression test case

selection process [19].

1.2 Overview of the Solution and the research challenges Improvements in software testing techniques are needed to address the increasingly

complex applications that are implemented by today’s software systems. The UML and

its diagrams are widely used to visually depict the static structure and more importantly

for us, the dynamic behavior of such applications. This trend provides us with an

excellent opportunity to meld our proven test generation technology (TDE) with the

UML and give developers and testers the capability of automatically generating black-

box conformance tests early on.

For regression testing, we can derive tests from UML Sequence diagrams, which embody

the behavioral description of each component of interest and more importantly, the

interactions or events between them. The Unified Modeling Language (UML) [14]

reduces the efforts in constructing the models in software development and increases

maintainability and reusability during the life cycle. Another well accepted method is

provided through design patterns. Likewise, frameworks provide users with an efficient

means to combine component reuse with design reuse.

Component based software engineering (CBSE) also offers convenience as well as fast

design/ implementation. Testing UML design models is challenging because a design

model consists of many models that describe different design views. An approach to

generating inputs for testing UML designs requires a mechanism that combines

information from both the structural and behavioral views of a design. We conducted the

case studies to investigate the effectiveness of our testing approach[3].

11

The studies include the effective way of generating the UML regression test case

selection based on UML sequence diagram. In my thesis, I have generated the storage

graph for sequence diagram using C++ program of data structure. The studies

demonstrate the cost of testing in terms of number of test inputs and effectiveness of test

case generation in terms of execution time.

Scope of the Research

The work described in my thesis focuses on effective way of generating the UML

regression test suite with the help of control flow graph (CFG) for the storage of the

sequence diagram. And the traversal of the control flow graph will automatically produce

the final regression test suite. The practical approach for generating the regression test

suite will provide more accuracy as compared to the manually generating the test suite.

The control flow graph for regression test selection techniques attempt to reduce the

time required to retest a modified program by selecting some subset of the existing test

suite. The purpose of RTS techniques is to reduce the high cost of retest-all regression

testing by selecting a subset of possible test cases.

In our approach, a control flow graph (CFG) will store the each events of the sequence

diagram.

They store the complete methods with parameters and information about the sender and

the receiver and with their pre-conditions and post-conditions. The specifications consist

of pre- and post-conditions expressed in the OCL.

1.3 Project Organization The rest of the thesis is organized as follows:

• Chapter 2 describe the Regression testing and its types.

• Chapter 3 presents Literature Survey of regression testing and defines the various

techniques of the regression testing.

• Chapter 4 describes the case studies of the existing Techniques.

• Chapter 5 discusses related work in component based regression testing and

UML.

• Chapter 6 Control flow graph (CFG) based criteria to generate regression test

suite.

12

• Chapter 7 describes our technique for generating the final regression test suite

from UML sequence diagram from different software component.

• Chapter 8 Conclusion and Future work.

13

Chapter 2

Regression Testing

Regression testing is a testing process, which is applied after a program, is modified. It

involves testing the modified program with some test cases in order to re-establish our

confidence that the program will perform according to the (possibly modified)

specification. In the development phase, regression testing may begin after the detection

and correction of errors in a tested program. A tested program is a program which has

been tested with a high quality test plan.

Regression testing is a major component in the maintenance phase where the software

system may be corrected, adapted to new environment, or enhanced to improve its

performance. Modifying a program involves creating new logic to correct an error or to

implement a change and incorporating that logic into an existing program. The new logic

may involve minor modifications such as adding, deleting, rewriting a few lines of code,

or major modifications such as adding, deleting [9].

2.1 The Need for Regression Testing Software inevitably changes however well conceived and well written it initially

may be. Operational failures expose faults to be repaired. Mistaken and changed

requirements cause the software to be reworked. New uses of old software yield new

functionality not originally conceived in the requirements. The management of this

change is critical to the continuing usefulness of the software. The new functionality

added to a system may be accommodated by the standard software development pro-

cases.

Regression testing [9] attempts to revalidate the old functionality inherited from the

old version. The new version should behave exactly as the old except where new

behavior is intended. Therefore, regression tests for a system may be viewed as partial

operational requirements for new versions of the system. Figure1 shows a typical

example of a sequence of time intervals during the life of a software system.

14

Figure 2.1 Regression Testing

2.2 Types of Regression Testing Two types of regression testing can be identified based on the possible modification of

the specification. Progressive regression testing involves a modified specification.

Whenever new enhancements or new data requirements are incorporated in a system, the

specification will be modified to reflect these additions. In most cases, new modules will

be added to the software system with the consequence that the regression testing process

involves testing a modified program against a modified specification [7].

In corrective regression testing, the specification does not change. Only some

instructions of the program and possibly some design decisions are modified. This has

important implications because most test cases in the previous test plan are likely to be

valid in the sense that they correctly specify the input-output relation. However, because

of possible modifications to the control and data flow structures of the software, some

existing test cases are no longer testing the previously targeted program constructs. The

corrective regression testing is often done after some corrective action is performed on

the software.

15

2.3 Differences between Testing and Regression Testing Most people assume that regression testing is a simple extension of testing. However, it

is not always the case. There are several major differences between these two processes.

Availability of test plan Testing begins with a specification, an implementation of the

specification and a test plan with test cases added during the specification, design and

coding phases. All these test cases are new in the sense that they have not been used to

exercise the program previously. Regression testing starts with a possibly modified

specification, a modified program and an old test plan which requires updating. All test

cases in the test plan were previously run and were useful in testing the program.

Scope of test The testing process aims to check the correctness of a program, including its

individual components (e.g., functions and procedures) and the interworking of these

components. Regression testing is concerned checking the correctness of parts of a

program. The portion of a program which is not affected by modifications need not be

retested. An interesting problem is to determine the affected portion of a program in an

efficient manner. A solution to this problem may be the use of retestable units.

Time allocation Testing time is normally budgeted before the development of a product.

This time can be as high as half the total product completion time. However, regression

testing time, especially time for corrective regression testing, is not normally included in

the total product cost and schedule. Consequently, when regression testing is done, it is

nearly always performed in a crisis situation. The tester is normally urged to complete

retesting as soon as possible and most often is given limited time to retest.

Development information In testing, knowledge about the software development

process is readily available. In fact, the testing group and the development group may be

the same. Even if an organization has a separate testing group, the testers can usually

query the developers about any uncertainty in the software. But in regression testing, the

testers most likely will not be the developers of the product. Since regression testing may

be done at a different time and place, the original developers may no longer be available.

This situation suggests that any relevant development information should be retained if

regression testing is to be successful.

16

Completion time The completion time for regression testing should normally be less than

that for testing since only parts of a program are being tested.

2.4 Similarities between Testing and Regression Testing Several aspects of regression testing are similar to that of testing. In particular, the

purposes and testing techniques used are almost the same.

Purposes: The purposes of testing and regression testing are quite similar. They both aim

to:

1. Increase one’s confidence in the correctness of a program.

2. Locate errors in a program.

Some additional goals of regression testing are to: preserve the quality of the

software; the modified software should be at least as reliable as its previous version; this

may be achieved in many ways; one possible method is to insist that the same structural

coverage is achieved by both versions of the software; ensure the continued operation of

the software; this is an important goal because some users may become dependent on the

software product, and software developers have a responsibility to continue to provide

the same service to users.

Testing techniques Since test cases in a test plan depend on the chosen testing

technique, the testing technique used by both testing and regression testing should be the

same if the regression testing process involves the reuse of test cases. If regression testing

were to involve a different testing technique, then it would be difficult to reuse the

existing test plan. Another reason for using the same testing technique is that it is easier

to evaluate the quality of two software products if they are tested by the same technique.

At the current state of the art, it is difficult to compare the relative test effectiveness of

two different testing techniques [7].

17

2.5 The Problem of Regression Testing The problem of regression testing [8] may be broken down into two subproblems: the

test selection problem and the test plan update problem. The test selection problem is

concerned with the design and selection of test cases to fully test a modified program.

Some test cases may be selected from those in the existing test plan, while others have to

be created based on modifications made to the program. The test plan update problem

deals with the management of a test plan as a program is undergoing successive

modifications. Certain test cases will become obsolete and new test cases must be added

to test the modified and new features of the software. The test plan update problem may

be defined as follows:

Test Plan Update Problem:

Given a program P, and its specification S, a test plan T for testing P, a program P’ which

is a modified version of P, and its specification S’, generate a test plan T’ for P’ from T,

P, S, F” and S’.

From this definition, we can break down the test plan update problem into two

subproblems:

1. Test case classification problem given a program P, and its specification S, a test

plan T for testing P, a program P’ which is a modified version of P, and its

specification S’, group the test cases in T into three mutually exclusive sets:

reusable, retestable, and obsolete.

2. Update the test plan by deleting the obsolete test cases, and adding in the new-

structural and new-specification test cases. Observe that before we can solve

subproblem (2), we (2) Update problem need a solution to the test selection

problem [15].

2.6 The Retest Approach In this section, we describe an approach for corrective regression testing. The Retest

approach [9] tries to reuse the test plan in a way that will reduce the amount of testing.

The previous test plan is used to assist in test selection, and is analyzed in order to

classify the various test cases; only a subset of the previous test plan is rerun. The Retest

regression testing process consists of two major phases: the test case classification phase

18

and the test plan update phase (see following Figure2.2). We next describe the major

components of this regression testing process.

Static Analyzer This component is static in the sense that no test case is executed. All

analysis is done using the change information and the information stored in the test plan.

The static analyzer uses this information to classify the existing test cases into two

classes: reusable and unclassified. The reusable test cases are those tests which do not

exercise any modified code, while the unclassified test cases are those which do. Another

function of the static analyzer is to compute the structural coverage measure achievable

from the reusable test cases based on the stored dynamic behavior of these test cases.

Both the coverage measure and the unclassified test cases are passed to the Dynamic

Analyzer.

Dynamic Analyzer

This component is dynamic because it actually executes some test cases in order to:

1) Test the program,

2) Satisfy the structural coverage criterion, and

3) Classify the unclassified tests into obsolete and retestable

test classes.

Figure 2.2 the Retest regression testing process for Corrective Regression Testing

19

The Dynamic Analyzer executes each test which is unclassified in an order implied

by the test-case-order. These test cases will be executed according to their ordering

within the test-case-order. Note that all specification-based tests will be run first and put

in the retestable class. They will not become obsolete because the specification is

assumed to be unchanged. Any structural test which does not increase the structural

coverage measure will be put into the obsolete test class. All others are grouped into the

retest able test class.

The Dynamic Analyzer will stop test case execution whenever the structural coverage

criterion is satisfied. Any remaining unclassified tests may be placed into the obsolete

class, or handled according to the testing guideline specified in the test an output of the

Dynamic Analyzer is the coverage measure achieved by both the reusable and the

retestable test cases. If this coverage measure is not satisfactory, then the Dynamic Tester

is invoked; otherwise the testing process enters the test plan update phase.

Dynamic Tester: This component is similar to the Dynamic Analyzer and is invoked only

when the structural coverage criterion is not satisfied by both the reusable and retestable

test cases. A major function of this component is to execute the new structural test cases

which are designed to exercise the modified code or to satisfy the structural coverage

criterion. Only the test cases which will increase the structural coverage measure will be

stored. Once the structural coverage criterion is satisfied, the Test Plan Updater is

invoked [9]

20

Figure 2.3 an operational overview of Retest

A major difference between the Dynamic Analyzer and the Dynamic Tester is that the

former involves the changing and deleting of existing information in the test plan, while

the latter involves the execution of new tests and the addition of new information to the

test plan. There are two advantages in running the unclassified tests before running the

new-structural test cases:

• Some unclassified tests will test the modified code and therefore they can reduce

the effort in designing new tests.

• Some unclassified tests may be used to assist in test generation.

21

Test Plan Updater: This component creates a new test plan from the reusable, retestable

and new-structural test cases. The objective is to generate an up-to-date test plan for the

next cycle of modifications and regression testing. Most of the information stored in the

new test plan have actually been collected by the Dynamic Analyzer and the Dynamic

Tester. An operational overview of the Retest process is presented in above Figure 3.2.

22

Chapter 3

Literature Survey

Regression testing aims at providing confidence that the modifications are correct

and have not adversely affected other parts of the program. Regression testing can be

progressive or corrective the former involves retesting major changes to the program’s

specification. The latter is performed on a specification that essentially remains

unchanged, so that only minor modifications, which do not affect the overall program

structure, require retesting. For regression testing, it would be costly to repeat the whole

set of test cases used in the initial development of the program and unreliable to choose a

random subset of these test cases [4].

Therefore, it is important to select a suitable subset of test cases that accomplish

the objectives of regression testing. The selection of suitable test cases can be made in

different ways, and a number of regression testing approaches and algorithms have been

proposed. A strategy based on input partitioning and cause-effect graphing of the

program specification has been described. Leung and White have suggested classifying

the initial test cases as reusable, retestable, obsolete, and adding new-structural and new-

specification test cases and, then, selecting test cases from some of or all five classes.

They have also proposed methods to limit integration regression testing to a small set of

modules.

4.1 Techniques of Regression Testing:

Gupta, Harrold, and Soffa’s proposed slicing algorithm which is data flow based. It

uses slicing to explicitly detect definition-use pairs that are directly or indirectly affected

by a program modification. A program slice consists of all statement segments, including

conditionals that might affect the values of variables in the modified statement/segment.

The advantage of this slicing approach is that no data flow history is needed, nor is the

recomputation of data flow for the entire program required after its modification [8].

Leung and White have proposed the concept of “firewall” to assist the tester in

integration regression testing. For our study’s purpose, we have implemented the

“firewall” concept at the segment (vs. module) level and hence the name adapted firewall

algorithm. Leung and White suggest building “firewalls” to confine integration

23

regression testing to a small set of modules rather than allowing it to spread to many

other modules. A “firewall” involves the modules that are modified and their direct

ascendants and direct descendants [8].

An optimal retesting approach based on a 0-1 integer programming problem

formulation has been proposed by Fischer and extended in and solved by natural

optimization in Some software tools, which are based on some of the afore-mentioned

techniques, have been constructed.

Agrawal, Horgan, and Krauser’s incremental algorithm which selects test cases from T

whose outputs may be affected by the modifications made to the program [8]. The

incremental approach exploits the following observations:

• Not all statements in the program are executed under all test cases.

• If a statement is not executed under a test case, it can not affect the program

output for that test case.

Even if a statement is executed under a test case, it does not necessarily affect the

program output for that test case. Every statement does not necessarily affect every part

of he program output. These observations can be used to determine execution, dynamic,

and then relevant slices for the test cases.

Rothermel and Harrold divide regression testing into two phases for the purpose of cost

analysis. During the preliminary phase, changes are made to the software, and the new

version of the software is built. During the critical phase, the new version of the software

is tested prior to its release to customers. They described regression testing techniques as

follows: “Given a program P, a modified version P’, and a set T of test cases used

previously to test P, regression analysis and testing techniques attempt to make use of a

subset of T to gain sufficient confidence in the correctness of P’ with respect to behaviors

from P retained in P’.” [14]

Regression testing is an active area of research. Most of the work involves white

box testing where code is available to determine the test cases. Agrawal et al. addressed

the problem of determining test cases in the regression test suite on which new and old

programs may produce different outputs [ 8] .

They refer the problem of determining regression test suite on which the modified

program may differ from the original program as the incremental regression test problem.

24

They proposed good approximate solution to this problem with execution slice technique

to save regression testing effort with the focus on finding the test cases on which the

original and the modified program differ.

Wu et al. proposed a regression testing technique that selects test cases by utilizing

static information from the analysis of the program structure and dynamic information by

tracing the function-calling sequence. Their approach is based on data-flow method with

optimization to handle OO features, such as inheritance, dynamic binding polymorphism

and message passing. They need to create Functional Calling Graph (FCG) of each test

case to trace the functional execution sequence.

Harrold et al. Proposed the use of component meta-content or metadata to support

regression testing of component-based software. Metadata stores additional information

along with the components. In this sense, it is similar to the version information we

require for components[8].

25

Chapter 4

Case studies of the existing techniques

4.1 Generating Regression Tests via Model Checking

Regression Testing is a maintenance activity that attempts to validate modified

software and ensure that modifications are correct and have not inadvertently affected the

software. Selective regression testing attempts to identify test cases for reuse in testing

modified portions of the system. Specification-based regression testing techniques select

reusable test cases from the original test suite with respect to formal specifications of the

system. One should make sure that the cost of selecting the regression tests is lower than

retesting with the entire original test suite.

In this implementation, our work is to check the ability of model checkers to

generate counterexamples, where each counterexample can be considered as a complete

test Sequence. The general idea of our work is to use the difference between two versions

of software specifications– the original one and the modified version. Our approach,

therefore, supports different test coverage criteria. Currently, our work allows users to

choose from a limited set of FSMbased test coverage criteria to generate regression test

suites, including state transition level (STL), syntactic trigger events level (SynL) and

semantic trigger events level (SemL).

Component

We are taking the example of GUI component. In this, we are comparing the

functionalities of two version of GUI; first one is window 98 notepad and second is

window XP notepad and possibly generates the test suit from these techniques.

We follow the approach, RTMC (Regression Testing via Model Checking), which relies

on the existence of two different specification models – Spec#2(XP notepad) is the

specification of a modified version of the same system described by Spec#1(98 notepad).

The idea is to use the difference between the two models to guide regression test case

generation.

26

98 notepad Spec1#

Notepad of window 98 Specifications File Menu Edit Menu Select 1 New to open new file Cut cut the

selected text

Find

Finds the specified text in a file and stop at first occurrence of text

2 Open to open existing file size should be less then 64kb

Copy

copy the selected text Find Next

Finds the next occurrence of text entered in find box

3 Save Save current file Paste paste the text at cursor position

4 Save as save a copy of current opened file

Undo revert the previous action

5 Page Setup Set the margins of page for printing including header & footer and layout of page

Time/Date

insert current date and time

6 Print print the current file or selected text

Word Wrap Wrap text to opened notepad windows margin, Horizontal scrollbar is off

7 Exit exit from notepad Delete delete the selected text

8 Set Font format the font

9 Select All select all text in a file

Table 4.1.1 Test suite of the Window 98 notepad

27

XP notepad Spec2#

File Menu

Edit Menu Format Menu

View Menu

1 New to open new file

Cut cut the selected text

Word Wrap

Wrap text to opened notepad windows margin, Horizontal scrollbar is off

Status Bar

Will Show status bar for this option word wrap option should be off

2 Open to open existing file size should be less then 64kb

Copy

copy the selected text

Font format the font

3 Save Save current file

Paste paste the text at cursor position

4 Save as

save a copy of current opened file

Undo

revert the previous action

5 Page Setup

Set the margins of page for printing including header & footer and layout of page

Time/Date

insert current date and time

File Menu

Edit Menu

6 Exit exit from notepad

Delete delete the selected text

28

Set Font format the font

Select All select all text in a file

Table 4.1.2 Test suite of window XP notepad

Comparator Comparator is basically used to compare the previous version i.e. window 98 notepad

with the modified version i.e. window XP notepad.

• Spec#1 represent the window 98 notepad

• Spec#2 represent window XP notepad

Following is basics steps of comparator:

Spec#1 and Spec#2 are fed into a comparator;

• The differences between the specifications based on different test coverage

criteria (Spec Diff) will be generated and used to derive properties

• Each specification have the two types of the properties:

Hold properties (Not changed properties): Each property that holds is considered not to

have changed.

Not hold properties (changed properties): Each property that does not hold (a Non-Hold

Prop) is considered to have changed

29

Figure 4.1.1 show the comparator.

30

1 98 Notepad XP Notepad 2 File Menu File Menu 3 New Hold Prop New 4 open Hold Prop open 5 Save Hold Prop Save 6 Save As Hold Prop Save As 7 Page Setup Hold Prop Page Setup 8 Print Hold Prop Print 9 Exit Hold Prop Exit 10 Edit Menu Edit Menu 11 Cut Hold Prop Cut 12 Copy Hold Prop Copy 13 Paste Hold Prop Paste 14 undo Hold Prop undo 15 Time/Date Hold Prop Time/Date 16 Word Wrap non hold Replace 17 delete Hold Prop delete 18 set Font non hold Goto 19 Select All Hold Prop Select All 20 Find 21 Find Next 22 Search Format Menu 23 Find non hold Word Wrap 24 Find Next non hold Font

Table 4.1.3 test suite of comparator

Based on diagram 4.1.1 we have the above test suits that show the Hold and Not hold

properties of 98 notepad and XP notepad.

Coverage Selector

Comparator discovers the differences between two versions of the specification. It

works together with the coverage selector which allows user to choose test criteria that

produce the Regression Test Specification (RT Spec),

Coverage selector has the following specification level:

a) State Transition Level

b) Syntactic Trigger

c) Semantic Trigger level

Following test suits display the working of above three level:

31

Figure 4.1.2 Coverage Selector

State

Transition Level

Syntactic

Trigger

Level

Event

Semantic

Trigger

Level Event

diff in

trigger

events

Events has been totally modified

1 Edit.replace Will

replace

the

selected

text with

Edit.set Font Format.font

Edit.wodwrap

search.Find

search.Find

Next

Format.wordWrap

Edit.Find

Edit.Find Next

32

the text in

clipboard 2 New Events

Added

View.status

Edit.Goto

Table 4.1.4 Test suite of Coverage selector based on above diagram

Test Generator Test Generator produces a set of counterexamples for each non-hold property.

Currently, Test Generator chooses the shortest counterexample from the list,

translating it into corresponding test case.

Test Case ID

Module/Interface/ Event

Test Case Title Test Case Objective Test steps Expected Result

1 Edit.replace replace text

Replaces the selected text with text given in find box

copy the text then select the text which is going to be replaced goto Edit->Replace

Selected text will be replaced by the copied text

2 Format.. Font font format the font Format menu->font

open a font dialog box with font,font style,size,script option

3 Format.wordWrap wrap

Wrap text to opened notepad windows margin, Horizontal scrollbar is off

Format menu->wordwrap

horizontal scroll bar will disappear

4 Edit. Find find Finds the specified text in a file Edit->find open a find dialog box

33

and stop at first occurrence of text

asking find what text box with ok cancel button,up/down and match case options. After typing the text and selecting ok button first instance of that text will be searched

5 Edit.Find Next find next Finds the next occurrence of text entred in find box Edit->Find Next

same find dialogbox will be opened to find next occurrence of that word if it reaches at the end of document and no matching word is found then alert bx will display no match found

6 View. status vstatus Will Show status bar view-> status bar

Will Show status bar for this option wordwrap option should be off

7 Edit.Goto egoto Goto to specific line edit->goto

goto dialog box will be opened and will ask for enter line number.For this wordwrap option should be off

Table 4.1.5 Test suite of Test Generator

Result RTMC are the concept of using a model checker as part of the regression test generation

tool. They support a specification description, and different test coverage criteria. Most

importantly, it offers a significant opportunity to reduce testing costs and automatically

produce regression tests from formal specifications with greater confidence

34

Chapter 5

Related Work in component-based Regression testing and UML In the current scenario, focus has shifted on using UML design models to perform testing

activity. We summarize existing work on generating test inputs from Regression Testing

based on UML models. Work on testing using the UML has likely to be focus on

generating requirements and criteria for code testing from UML diagrams.

In Regression Testing UML design of software components we identify changes to UML

designs to assist in classifying UML test cases. The UML designs we consider consist of

UML Class Diagrams, Sequence Diagrams, and OCL expressions. The one formal

implicit in our approach is that we must rely on the integrated model. In UML integrated

model, The Class Diagram consists of the following primary elements: classes, attributes,

methods, and associations. The Sequence Diagram contains: objects, lifelines, conditions

and messages. Each of these elements can be changed by addition, deletion, or

modification.

Scheetz et al. [20] Towards automated support for deriving test data from UML

sequence diagrams paper describes an approach to generate system test inputs from

UML class diagrams. The class diagrams are restricted to contain only classes,

associations and specifications and specification structures. In this approach we first

identify test objectives for every single class. A test objective describes a set of objects in

terms of the states they can take on. Test objectives are derived from defining desired

states of the class instances after the test is executed. A state of an object is defined based

on its attribute values and links to other objects. Test objectives for a complete class

diagram can be aggregated from test objectives for each individual class specified in the

class diagram.

In this approach, we used the UML and its associated object constraint language (OCL)

to describe the information about the changes from one version to next which was

provided by component developer and comprised modified method and

directly/indirectly affected methods, then utilized this information to direct the test case

selection. This method is quite simple and can be easily implemented in practice, but this

method is so rough that the selected subset of test cases generally contains many

irrelevant test cases

35

Pilskalns et al. [15] Regression testing design paper describes the purpose of a Graph-

based approach to combine the information from structural and behavioral diagrams

(Class Diagram and Sequence Diagrams). In this approach, each sequence diagram is

transformed into an Object-Method Directed Acyclic

Graph (OMDAG). Each node in an OMDAG represents a method call or a return action,

as well as the class of the object that initiates the call. The directed arcs represent control

flow. OMDAGs can be used to generate the execution paths in the sequence diagram that

satisfy different test adequacy criteria. UML testing technique proposed to define

regression testing for UML designs.

UML testing relies upon building an integrated model from Class Diagrams, Sequence

Diagrams and OCL. Building the integrated model consists of three steps. The first step

maps Class Diagrams into tuples, called Class Tuples (CT). The CT consists of a class

name, attributes (represented as tuples if non-primitive) from the class and super classes

(if applicable), and methods (represented as tuples) for the class and super classes (if

applicable). A class tipple is a mathematical representation of a class and is similar to the

idea of representing a class using the XMI specification. OCL statements are mapped into

the CT based upon their context.

The second step consists of mapping Sequence Diagrams into an Object Method Directed

Acyclic Graph (OMDAG). The OMDAG maps the dynamic information in a Sequence

Diagram to a directed acyclic graph. The OMDAG is created by mapping object and

sequence method calls from a Sequence Diagram to vertices and arcs in a directed acyclic

graph.

They propose a regression testing technique based on UML sequence and class diagrams.

Their approach does not take into account the pre and post conditions of the operations

which affect behavior of a class. Also, their approach does not handle concurrency.

So finally we have concluded that there is some lack of points in the existing approach.

These points are as follow:

1. This method is quite simple and can be easily implemented in practice, but this

method is so rough that the selected subset of test cases generally contains many

irrelevant test cases.

36

2. It’s difficult to compute the efficiency from manually UML based test case

generation.

3. Above discuss approach does not take into account the pre and post conditions of

the operations which affect behavior of a class. Also, their approach does not

handle concurrency.

In our work, we would require to extend the work of manual UML based Regression

testing and will generates efficient UML regression test case generation for software

component by using programmatically Graph-Based approach.

37

Chapter 6 Control Flow Graph (CFG) based criteria to generate Regression Test Suite. Our approach is to generate Regression Test suite for software components is based on

the UML sequence diagram storage graph.

Figure 6.1 shows the overall working of the control flow graph based criteria for the

generation of the regression test suite.

We construct the storage graph using C++ program of data structure. We have used two

flow graphs; control flow graph CFG and extended current flow graph ECFG to store the

UML sequence Diagram of the software components. The basic information of sequence

diagram is sequence of events that are extracted from the diagram. The UML sequence

diagram designed with the help of Net Beans Tool. Net Beans Unified Modeling

Language (UML) tools enables you to generate UML Diagrams such as sequence

diagrams. A net beans tool is the java enabled tool, there is no need to drawn the

38

sequence diagram manually using the net beans tool, it works as reverse engineering tool.

It takes the java program source code and generates the sequence diagram automatically.

The two flow graphs are compared to identify the changes and hence to generate test

cases. The different steps of our approach are as follows:

1. Built Sequence diagrams for software components.

2. To construct the CFG to store sequence diagram.

3. Traversal of the Storage graph to generate regression test suites.

1. Built Sequence diagram for software components: Sequence Diagrams: A Sequence Diagrams models system behavior by specifying how

object interact to complete the task. An interaction is expressed by the messages between

lifelines. A lifeline is participant in an interaction. A lifeline represents a class instance.

In this thesis, a message can represent a message invocation, a reply message, a creation

and deletion of a class instance.

Figure 6.2 shows the Basic Sequence Diagrams of Students who are enrolling in

seminar in University.

2. To construct the Control Flow Graph (CFG) to store sequence

diagram. In this part of our approach we construct the control flow graph( CFG) using the C++

program of data structure. The minimum requirement for the storage graph is window 98

39

or higher version and minimum 32 MB RAM. In the control flow graph the States are

represented by vertices of graph and methods by edges through which the object changes

from one state to another. The precondition and postcondition must be satisfied all the

methods and their parameter are stored as edges. The construction Storage Graph is built

using Graph of Data Structure, Where each Edges and Vertices are represented as State.

Struct EdgeState; Struct VertexState; Struct VertexState { Char StateId [10]; int IsInitial, IsFinal; VertexState *NextVertex; EdgeState *EdgeRoot; VertexState () { IsInitial=IsFinal=0; EdgeRoot=NULL; } The composing of vertex node is as follows:

State Name Next State Method

Pointer

Precondition Postcondtion

The Edge node is as follows:

Method

Id

State

From

State To Sender Receiver Attributes Next

method

Each attribute is abstract type and consists of:

Date Type Attribute Name

Algorithm for the generation of storage graph for the sequence

diagram: A state is to be added in the sequence diagram. A graph in the sequence diagram and

states represent the vertices of the graph. INIT is the initial state; if it is NULL then we

can say that the sequence diagram is missing.

40

if INIT = NULL then

Display “No diagram found”

Endif

To add a state S into graph G1 representing sequence diagram of first version, we will

use it as:

if INIT = NULL then INIT = S

Display “first assume intail state is empty then add in it one state.”

// Last state is extracted with the help of a loop traversing each state of a graph

Else t = G1.lastState then t.NextState= S

Endif

Adding methods/ transitions into States of the control flow graph. A method represents the transition from state s1 to state s2 in graph. It has many

attributes like methodid, parameters, sender, receiver, and return type of the method etc. Let s1 = source state of the transition

s2 = destination state of the transition

M = method of the of the each transition

x = G1.findsourceState (s1)

If X = NULL then

Display “invalid source state”

Endif

End

t1 = x.lasttransition ( ) // list transition in the list of transition as from state x

will be extracted

t1-> NextTransition = M // new method will added in the transition list.

3. Traversal of the storage graph to generate regression test suites.

After written of the control flow graph we compare both version of the sequence

diagram. And traverse the storage graph for both the version of the sequence diagram.

We used depth first search technique for the traversal of the regression test case

41

algorithm. And based on the comparison of the storage graph, we generate the

regression test suites.

G1= Graph for first Sequence Diagram

G2= Graph for Modified Version of G1

T1= Traverse Stats of G1

T2= Traverse States of G2

E1= Edges of the Graph G1

E2= Edges of the Graph G2

1. T2= G2[INIT] // initial states of Graph G2

2. while T2 != NULL // repeat for all states of graph while their exist a transition

E2= T2 [METHOD POINETR]

While E2 ! = NULL // repeat for all transition of current states i.e. for all

methods that is applicable at current state

CALL Check Match (G1, E2) // search the method E2 in G1

E2=E2 [NEXTMETHOD] // jump to next transition/method

End while

T2=T2 [NEXTSTATE] // jump to next state

End while

3. End

// ChekMatch method

CheckMatch (G1, E2) // this function match the method E2 with in

sequence diagram G1

1. T1= G1[INIT]

Match=false // assume match is not present

While (T1 != NULL) // start search match for each states and each method

within state.

E1= T1 [METHOD POINETR]

While (E1 != NULL)

42

// all attributes of E1 and E2 matches

If E1=E2 then

Display “Reusable”

Return

// if method name match but attributes do not mark as true

Else if E1 [MethodId] = E2 [MethodId]

Match= True

Endif E1= E1 [NEXT METHOD]

End while

// if match is true that is method was found but attributes did not matched that method is

invalid

If Match= True then Display “Obsolete”

Else if Match= false

// if nothing matches not even method name then it newly introduced method

Display “test case are newly added or ReTestable”

43

Chapter 7 Case Studies

In the previous chapter we have constructed an algorithm of control flow graph for the storage of sequence diagrams. In which, we created the states and transition for the graph. And after that, we have written an algorithm for the traversal of the storage graph for the generation of the regression test suites. Now these algorithms will be practically experimented on actual components and the component is drawn with the help of Net Beans Tool. In this chapter, we have generated the regression test cases using the sequence diagrams of the three different components. 7.1 Case Study with ATM using the Net Beans Tools for UML Sequence

Diagram a) The algorithm for constructing the storage graph will store all possible information of

ATM into the States and Edges, before that:

First we generated the Sequence Diagram of both the version using Net Beans tools.

Figure 7.1.1 shows the first version of the ATM Sequence Diagram..

And with help of the algorithm of storage graph for the sequence diagram. We store

each states and transitions of the ATM sequence diagram.

Source Destination Method &Parameter Sender Receiver

44

ST1

ST2

ST3

ST3

ST4

ST2

ST1

ST2

ST4

ST3

Request(int amount, int AccNo)

getcash(int amount, string recpt)

Invalid(int AccNo, String Msg)

verfybal(int amount, int AccNo)

balinfo(int bal, int AccNo)

USER

ATM

CONSORT

CONSORT

BRANCH

ATM

CONSORT

ATM

BRANCH

CONSORT

Table 7.1.1 shows the case scenarios based on the ATM sequence diagram of first

version.

The scenarios show all the transition in which we have a different source and

destinations, methods, sender and receiver. We traverse the graph and store all the

information from UML Sequence diagram.

45

Figure 7.1.2 show extending the functionality of first version of ATM sequence

Diagram.

The modified version of the ATM shows some extra functionality from the previous

version of the ATM. The modified component has some new events such as states

and transition which was not in the previous version of the sequence diagram.

Source Destination Method &Parameter Sender Receiver

46

ST1

ST2

ST3

ST3

ST4

ST4

ST5

ST5

ST5

ST2

ST1

ST2

ST4

ST3

ST3

ST4

ST4

ST2

Request(int amount, int AccNo)

getcash(int amount, string recpt)

Invalid(int AccNo, String Msg)

verfybal(int amount, int AccNo)

balinfo(int bal, int AccNo)

clear(int AccNo, int amount)

failure(int AccNo, string Msg)

Success(int balance, int Recptno)

Recpt(int prvbal, int currbal)

USER

ATM

CONSORT

CONSORT

BRANCH

BRANCH

CASH_ST

CASH_ST

CASH_ST

ATM

CONSORT

ATM

BRANCH

CONSORT

CASH_ST

BRANCH

BRANCH

ATM

Table 7.1.2 shows the nine case scenarios based on the ATM sequence diagram of

modified version.

b) Based on the case scenarios of the both the diagrams, the algorithm of traversal

will compares the events of the both the version. And generates the regression test

suites. The algorithm of Regression Testing will compare both the version of the

sequence diagrams or interaction diagrams. So that we can find out how many test

cases are same in both the version and how many tests are new in modified

version, and how many tests are not valid.

TEST ID STATE METHOD (PARAMETER) SENDER RECEIVER TEST ID : 01 ST2 Request(int amount, int AccNo) USER ATM Reusable ------------------------------------------------------------------------------------------------------------------------------------

47

TEST ID : 02 ST1 getcash(int amount,string recpt) ATM USER Reusable -------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 03 ST3 vldacc(int AccNo,int cardno) CONSORT BRANCH Obsolete ---------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 04 ST2 invalid(int AccNo,string msg) CONSORT ATM Reusable --------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 05 ST4 verfybal(int 0.0000,int AccNo) CONSROT BRANCH Obsolete ----------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 06 ST3 balinfo (int cbal,int AccNo) BRANCH CONSORT Reusable ------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 07 ST5 clear(int AccNo,int amount) BRANCH CASH_ST NewAdded -------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 08 ST4 failure(int AccNo,string msg) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 09 ST4 success(int balance,int recptno) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 10 ST2 recpt (int prvbal, int currbal) CASH_ST ATM NewAdded ---------------------------------------------------------------------------------------------------------------------------------------------- Table 7.1.3 shows the final Regression Test case generation from the ATM component.

In regression test suites, we find the changes occur in the modified version of the ATM

component. In the regression test suites half of the test cases are from the modified

component. So they all are new added.

6.2 Case Study with E-Ticket of Cinema for UML Sequence Diagram Our second case study with the E-Ticket of Cinema for movie, we have generated the

sequence diagram first for both the version of the component. In the E-Ticket of Cinema,

the sequence diagram describes how the customer will login for ticket, and after login

they can able to see the detail of the movies. They can choose the movie from the

available list.

48

In our first step, we have generated the sequence diagram for both the version of Cinema

component,

Figure 7.2.1 shows the sequence diagram of first version of E-Ticket of Cinema

component. Source Destination Method(parameter) Sender receiver

S1 S2 Login(string Un) Customer Boooking

S1 S2 TickDet(int TickNo,int seat)

Customer Boooking

S2 S1 LoginFail() Booking Customer

S2 S3 ChFilm(string name) Booking Movie

S2 S4 ChCinema() Booking Custom

S2 S5 ChSeat(int seatNo) Movie SeatPlan

S3 S2 AvailFilm(list film) Movie Booking

S3 S4 GetMovie() Cinema Cinema

49

S4 S3 Details(list films) Movie Movie

S5 S2 SeatAlot(list SeatNo) Seatplan Booking

Table 7.2.1 shows the Case scenarios of first version of the E-Ticket system of Cinema. This case scenario is generated with the help of the storage graph algorithm; we traverse

each states and transition of the graph. And in the transition we have various sender and

receiver, method and their parameters.

Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema. Source Destination Method(parameter) Sender receiver

S1 S2 Login(string Un) Customer Boooking

S1 S2 TickDet(int TickNo,int seat)

Customer Boooking

S2 S1 LoginFail() Booking Customer

S2 S3 ChFilm(string name) Booking Movie

50

S2 S4 ChCinema() Booking Custom

S2 S5 ChSeat(int seatNo) Movie SeatPlan

S3 S2 AvailFilm(list film) Movie Booking

S3 S4 GetMovie() Cinema Cinema

Source Destination Method(parameter) Sender Receiver

S4 S3 Details(list films) Movie Movie

S5 S2 SeatAlot(list SeatNo) Seatplan Booking

S5 S6 Showstat() Seatplan Seats

S6 S5 Status() Seats Seatplan

S1 S6 Fill_form() Customer InpForm

S6 S1 Validform() InpForm Customer

Table 7.2.2 shows the Case scenarios of modified version of the E-Ticket system of

Cinema.

Based on the comparison of the both the control flow graph of first version of the E-Ticket of Cinema and Extended flow graph of the modified version of the E-Ticket of Cinema. The traversal of the regression test algorithm will produce the final regression test suite. Test_ID State Method(Parameter) Sender Receiver Comments

01 S2 Login(string Un) Customer Booking ReUsable

02 S2 TickDet(int TickNo,int seat)

Customer Booking ReUsable

03 S7 Fillform() Cinema InpForm New Added

04 S1 LoginFail() Booking Customer ReUsable

05 S3 Choose(string name) Booking Movie New Added

06 S4 ChCinema() Booking Cinema Obsolete

51

07 S5 ChSeat(int seatNo) Booking Seatplan ReUsable

08 S2 AvailFilm(list film) Movie Booking ReUsable

Test_ID State Method(Parameter) Sender Receiver Comments

09 S4 GetMovie Movie Cinema ReUsable

10 S3 Details (list films) Cinema Movie ReUsable

11 S2 SeatAlot(int SeatNo) Seatplan Booking Obsolete

12 S6 ShowStat() Seatplan Seats New Added

13 S5 Status() Seats Seatplan New Added

14 S1 ValidForm() InpForm Customer New Added

Table 7.2.3 shows the final Regression Test case generation from the E-ticket of cinema

component.

7.3 Case Study with Library Record System for UML Sequence

Diagram Now practically experiment our approach with third component i.e. Library Record System,

52

a) Based upon the steps, first generates the sequence diagram for both of the library record system. Then possible stores those into the storage graph. And then traversal of the storage graph for regression test case selection.

Figure 7.3.1 shows the Sequence diagram of the first version for the Library Record system

Source Destination Method(parameter) Sender receiver

S1 S2 FillForm(int form_no,int AmtFee)

Person membship

S2 S3 ReqIssue(int BookNo,int BName)

Membship Librarian

S2 S1 Verify(string PerName,string Address)

Person Membship

S2 S1 Register(int PerId,string msg)

Membship Person

S2 S3 RetBook(int BookNo,int Date)

Membship Librarian

S3 S2 Issued(int MembId,int BookNo)

Librarian membship

53

S3 S2 Fine(int delay, float amount)

Librarian membship

S3 S4 Request(list order,float cost)

Librarian PurDept

Table 7.3.1 shows the case scenarios of the first version of the Library Record System

Figure 7.3.2 shows the Sequence diagram of the modified version for the Library Record system. Source Destination Method(parameter) Sender receiver

S1 S2 FillForm(int form_no,int AmtFee)

Person membship

S2 S3 ReqIssue(int BookNo,int BName)

Membship Librarian

S2 S1 Verify(string PerName,string Address)

Person Membship

S2 S1 Register(int PerId,string msg) Membship Person

S2 S3 RetBook(int BookNo,int Membship Librarian

54

Date)

S3 S2 Issued(int MembId,int BookNo)

Librarian membship

S3 S2 Fine(int delay, float amount) Librarian membship

Source Destination Method(parameter) Sender receiver

S3 S4 Request(list order,float cost) Librarian PurDept

S4 S5 Order(list olderls) PurDept BookDepo

S4 S5 IsAvail(list books) PurDept BookDepo

S5 S4 Status() BookDepo PurDept

S5 S4 Bill(float amount) BookDepo PurDept

Table 7.3.2 shows the case scenarios of the modified version of the Library Record

System.

b) Based on the comparison of the control flow graph of first version of the library record

system and extended flow graph of the modified version of the library record system. The

traversal of the regression test algorithm will produce the final regression test suite.

Test_ID State Method(parameter) Sender Receiver Comments

01 S2 FillForm(int formno ,int

AmtFee)

Person Memship ReUsable

02 S3 ReqIssue(int BookNo, int

BName)

Memship Librarian ReUsable

03 S1 Verify(string

PerName,string Address)

Memship Person Obsolete

55

04 S1 Register(int PerId, string

msg)

Memship Person ReUsable

Test_ID State Method(parameter) Sender Receiver Comments

05 S3 RetBook(int

BookNo,string Date)

Memship Librarian Obsolete

06 S2 Issued(int MembId, int

BookNo)

Librarian MembShip ReUsable

07 S2 Fine(int delay, float

amount)

Librarian MembShip ReUsable

08 S4 Request(list order, float

cost)

Librarian Purdept ReUsable

09 S5 Order(list olderls) Purdept Bookdepo New Added

10 S5 IsAvail(list books) Purdept Bookdepo New Added

11 S4 status () Bookdepo Purdept New Added

12 S4 Bill(float amount) Bookdepo Purdept New Added

Table 7.3.3 shows the final regression test suite for Library record system

56

Chapter 8

Conclusions and Future work

8.1 Evaluation of case studies: a) Evaluation of the ATM component

We can check the running time of the ATM component for producing the final regression

test suite. This can be checked with the help of the bar chart.

New ATM test case File

Final Regression Test suite

Old ATM test case File

Component: ATM

Total regression test cases Reusable Obsolete Retestable 10 4 2 4

10

42

4

0

5

10

15

Series1 10 4 2 4

Total ReUsable Obsolete Retestable

Running Time: 0.011

Figure 8.1.1 shows the Bar chart of the total number of regression suite for ATM

component

The bar chart shows the total number of the test cases and the shows how many test cases are Retestable, Reusable, and Obsolete in regression testing along with their running time.

57

b) Evaluation of the E-Ticket of Cinema component

We can check the running time of the E-ticket of Cinema component for producing the

final regression test suite. This can be check with the help of the bar chart.

.

Component: E-Ticket of Cinema

Total regression test cases Reusable Obsolete Retestable

14 7 2 5

New E-ticket of Cinema test file

Final Regression test suite

Old E-ticket of Cinema test file

14

7

2

5

0

5

10

15

Series1 14 7 2 5

Total ReUsable Obsolete Retestable

Running time: 0.0264

Figure 8.1.2 shows the total number of regression suite for E-ticket of Cinema component

The bar chart shows the total number of the test cases and the shows how many test cases

are Retestable, Reusable, and Obsolete in regression testing along with their running

time.

58

c) Evaluation of the Library Record System component

We can check the running time of the Library Record System component for producing

the final regression test suite. This can be check with the help of the bar chart.

Final Regression test suite

Component: Library Record System

Total regression test cases Reusable Obsolete Retestable 12 6 2 4

New Library Record System test file

Final Regression test suite

Old Library Record System test file

12

6

24

0

5

10

15

Series1 12 6 2 4

Total ReUsable Obsolete Retestable

Running Time: 0.0247

Figure 8.1.3 shows the total number of regression test suite for Library record system

component.

The bar chart shows the total number of the test cases and the shows how many test cases

are Retestable, Reusable, and Obsolete in regression testing along with their running

time.

59

8.2 Summary of the contribution.

Regression testing is defined as the execution of a set of application tests to verify that

the application’s quality has not degraded. Regression testing is necessary after a change

to the application program, a part of the application to a different vendor’s machine, or

change in the underlying operating system. Regression testing techniques generally apply

to test modified code, not designs.

We have proposed a systematic technique of UML based regression testing applied on

sequence diagrams. The primary goals in the design of the UML are to provide users with

a ready-to-use, expressive visual modeling language so they can develop and exchange

meaningful models. The UML Sequence diagrams are the most popular UML artifact for

dynamic modeling, which focuses on identifying the behavior within your system.

We started with the manual Regression testing for UML design. We have performed

black-box as well as white box testing for the generation of the UML based regression

test suite. As a part of our research work, programs have been written in C++ to generate

and traverse control flow graph for the storage and traversal of UML sequence diagram.

The sequence diagrams of different versions of software components are initially stored

and then compared automatically to generate regression test suite.

We have practically demonstrated the approach by considering three different case

studies from different application domains. The results have been verified by the running

time of the regression test suite generation technique.

8.3 Future Work Currently we have implemented our programs in C++ to construct control flow graph for

the storage of the UML sequence diagrams and they have been traversed automatically

for generation of the regression test suite. So we are only using the UML sequence

diagrams for the generations of the final regression test suite. In future, to generate the

regression test suite; other UML diagrams like activity and use case models can also be

considered. It will widen the domain of analysis of the software component. Thus,

resulting in effective test suite.

60

References

[1] Atifah Ali1, Aamer Nadeem1, Muhammad Zohaib Z. Iqbal2, Mohammad Usman,

“Regression Testing based on UML Design Models”, in the proceeding of the 10th

International Conference on Information Technology, IEEE press 2007.

[2] Bruce W. Weide, "Modular Regression Testing", Connections to Component-Based

Software, Dept. of Computer and Information Science, The Ohio State University , 2015

eil Ave. Columbus, OH 43210,USA +1 614 292 1517 weide.1@osu.edu , IEEE press

2005 .

[3] Chengying Mao, Yansheng Lu,” Regression Testing for Component-based Software

Systems by Enhancing Change Information”, Proceedings of the Asia-pacific Software

Engineering Conference (APSEC’05), Document number 0-7695-2465-6/05, IEEE press

2005.

[4] David Binkleyf “Reducing the Cost of Regression Testing by Semantics Guided Test

Case Selection”, Proceedings of the Department of Computer Science Department

Loyola College in Maryland 4501 North Charles Street Baltimore, Maryland 21210

binkley @cs.loyola.edu, IEEE press 1995.

[5] Demissie B. Aredo “Sematics of UML Sequence Diagrams in PVS”, in the

proceeding of the Department of Informatics, University of Oslo Institute for Energy

Technology.

[6] Fevzi Belli, Christof J. Budnik, “Towards Self-Testing of Component-Based Software

“, Proceeding of the 29th Annual International Software and Applications Conference

(COMPSAC ’05), IEEE, 2005.

61

[7] Gregg Rothermel and Mary Jean Harrold “A Safe, Efficient Algorithm for

Regression Test Selection”, Proceedings of the Department of Department of Computer

Science Clemson University Clemson, SC, 29634-1906, {grot her,

harrold}@cs.clemson.edu, IEEE press 1993.

[8] Ghinwa Baradhi and Nashat Mansour “A Comparative Study of Five Regression

Testing Algorithms”, in the procededing of the Department Ghinwa Baradhi and Nashat

Mansour Computer Science Lebanese American University, nmansour@lau.edu. Lb,

IEEE press 1997.

[9] Hareton K. N. Leung “Insights into Regression Testing”, Proceedings of the

Department of Computing Science University of Alberta Edmonton, Alberta, Canada

T6G 2H1, IEEE press 2000.

[10] Jiang Zheng1, Laurie Williams1, Brian Robinson2, Karen Smiley “Regression Test

Selection for Black-box Dynamic Link Library Components”, Proceedings of the

Department of Computer Science, North Carolina State University, Raleigh, NC 27695

{jzheng4, lawilli3}@ncsu.edu 2 ABB Inc., US Corporate Research {brian.p.robinson,

karen.smiley}@us.abb.com, IEEE press 2007.

[11] L. C. Briand Y. Labiche G. Soccar “Automating Impact Analysis and Regression

Test Selection Based on UML Designs” , Software Quality Engineering Laboratory

Carleton University Department of Systems and Computer Engineering 1125 Colonel By

Drive Ottawa, ON, K1S5B6, Canada {briand} labiche}@sce.carleton.ca, IEEE press

2002.

[12] Li Bao-Lin, Li Zhi-shu, Li Qing, Chen Yan Hong “Test Case automate Generation

from UML Sequence diagram and OCL expression”, 15th International Conference on

Advanced Computing and CommunicationsChina, IEEE 2007.

62

[13] Lihua Xu, Marcio Dias, and Debra Richardson “Generating Regression Tests via

Model Checking” Proceedings of the Department of Informatics, School of Information

and Computer Science, University of California at Irvine, CA, USA {lihuax, mdias, djr}

@ ics.uci.edu, IEEE press 2004.

[14] Monalisa Sarma and Rajib Mall “Automatic Test Case Generation from UML

Models”, in the proceeding of the 15th International Conference on Advanced

Computing and Communications , IEEE press 2007.

[15] Orest Pilskalns and Gunay Uyan “Regression Testing UML Designs”, Proceedings

of the Washington State University Vancouver, WA orest,gunay@vancouver.wsu.edu,

IEEE press 2006.

[16] Orest Pilskalns, Daniel Williams, Damir Aracic “Security Consistency in UML

Designs”, in the proceeding of the School of Engineering and Computer Science

Washington State University, Vancouver, WA 99686, IEEE press 2006.

[17] Rami Bahsoon and Nashat Mansour “Methods and Metrics for Selective Regression

Testing”, Proceedings of the Department of Computer Science, University College

London, Gwer Street, London, WClE 6BT, UK. E-mail: r.bahsoon @cs.ucl.ac.uk, IEEE

presses 2001.

[18] Sujit Kumar Chakrabarti, and Y. N. Srikant “Specification Based Regression Testing

Using Explicit State Space Enumeration”, Proceedings of the International Conference on

Software Engineering Advances (ICSEA'06), IEEE press 2006

[19] Siros Supavita and Taratip Suwannasart “Testing Polymorphic Interactions in UML

Sequence Diagrams”, in the proceeding of the Department of Computer Engineering,

Faculty of Engineering Chulalongkorn University, Bangkok,IEEE press 2007.

63

[20] Scheetz et al. and j.cui “Towards Automated Support For Deriving Test Data From

UML Sequence Diagrams”, in the proceeding of the 6th international conference on the

UML. San Franscisco, CA, USA, October 2003.

[21] Weiqun Zheng and Gary Bundell “Model-Based Software Component Testing: A

UML-Based Approach”, in the procededing of the Department Weiqun Zheng and Gary

Bundell Centre for Intelligent Information Processing Systems, School of Electrical,

Electronic and Computer Engineering, , IEEE press 2007.

64