4/25/2015CPSC-4360-01, CPSC-5360-01, Lecture 111 Software Engineering, CPSC-4360-01, CPSC-5360-01,...

64
03/13/22 CPSC-4360-01, CPSC-5360-01, Lecture 11 1 Software Engineering, CPSC-4360-01, CPSC-5360- 01, Lecture 11

Transcript of 4/25/2015CPSC-4360-01, CPSC-5360-01, Lecture 111 Software Engineering, CPSC-4360-01, CPSC-5360-01,...

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 1

Software Engineering, CPSC-4360-01, CPSC-5360-01, Lecture 11  

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 2

Review of Last Lecture Implementing a Statechart Review of Java programming concepts

Inheritance Polymorphism Type Casting Interface

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 3

Overview of This Lecture

Software Testing Overview Test Phases

Unit Testing Integration Testing

OO Specific Testing Automated Test Driver

Overview Example

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 4

Where are we now?

Requirement

Analysis

Design

Implement

Test

Evaluating the System

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 5

Testing – Famous Quotes

“Testing is the process of comparing the invisible to the ambiguous, so as to avoid the unthinkable happening to the anonymous.”, James Bach

"Testing is organised skepticism.“, James Bach

"Program testing can be used to show the presence of bugs, but never to show their absence!“, Edgar Dijkstra

"Beware of bugs in the above code; I have only proved it correct, not tried it.“, Donald Knuth

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 6

Testing: Definitions

1. “Testing is the process of establishing confidence that a program or system does what it is supposed to.”

2. “Testing is the process of executing a program or system with the intent of finding errors.”

3. “Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.”

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 7

Testing: Overview

1. A software test process is based on well-defined software quality control and testing standards, testing methods, strategy, test criteria, and tools.

2. Engineers perform all types of software testing activities to perform a software test process.

3. The last quality checking point for software is on its production line.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 8

Testing: Objectives

1. Uncover as many errors (or bugs) as possible in a given time.

2. Demonstrate that a given software product matches its requirement specifications.

3. Generate high quality test cases, perform effective tests, and issue correct and helpful problem reports.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 9

Testing: Objectives

4. Uncover the errors (defects) in the software, including errors in:

1. requirements from requirement analysis.

2. design documented in design specifications.

3. coding (implementation).

4. system resources and system environment.

5. hardware problems and their interfaces to software.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 10

Testing: Personnel 1. Test Manager:

1. Manage, supervise, and control a software test project.2. Define and specify a test plan.

2. Software Test Engineers and Testers:1. Define test cases, write test specifications, run tests.

3. Development Engineers:1. Only perform unit tests and integration tests.

4. Quality Assurance Group and Engineers:1. Perform system testing.2. Define software testing standards and quality control

process.

5. Independent Test Group.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 11

Testing: Activities

1. Test Planning:1. a test schedule for a test process and its activities,

as well as assignments.

2. test requirements and items.

3. test strategy and supporting tools.

2. Test Design and Specification:1. conduct software design based on well-defined

test generation methods.

2. specify test cases to achieve a targeted test coverage.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 12

Testing: Activities

3. Test Set up:1. Testing Tools and Environment Set-up.

2. Test Suite Set-up.

4. Test Operation and Execution:1. Run test cases manually or automatically.

5. Test Result Analysis and Reporting:1. Report software testing results and conduct test result

analysis.

6. Problem Reporting:1. Report program errors using a systematic solution.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 13

Testing: Activities

7. Test Management and Measurement:1. Manage software testing activities, control testing schedule,

measure testing complexity and cost.

8. Test Automation:1. Define and develop software test tools.

2. Adopt and use software test tools.

3. Write software test scripts.

9. Test Configuration Management:1. Manage and maintain different versions of software test

suites, test environment and tools, and documents for various product versions.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 14

Testing: Cycle of Activities

Testing usually involves repetition of testing activities. Example:

TestCases

PerformTest

LocateError

Design Error Repair

RepairError

TestResults Specification

retest

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 15

Test Organization In a large software system, testing has to be

carried out in multiple stages. The test stages are linked to the various

software development stages. Example:

The Acceptance Test is linked to the Requirement and the Specification stages.

Also known as the V-Model, it refers to the next slide to see the alphabet ‘V’ in the diagram.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 16

Testing Organization (V-Model)

Produce ValidateRequirements specification

System Specification

System Design

Module Design

Coding

UnitTesting

IntegrationTesting

System Testing

Acceptance Testing

UserSpecification

SystemSpecification

SoftwareSpecification

ModuleSpecification

Produce Validate

Produce Validate

Produce Validate

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 17

Test Stage: Unit Test Testing individual modules:

Methods or Functions. Classes. Sometimes class clusters.

It is based on the information about the structure of a code fragment.

Carried out by the Development Engineer. The objective is to test that the unit performs the

function for which it is designed. Unit tests can be designed before coding begins,

or just after the source code is generated.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 18

Unit Test: Check List Interface:

To ensure information properly flows in and out. Data structure:

To ensure the stored data maintains integrity. Boundary conditions:

Module operates properly at boundaries. Independent paths:

All paths through the control structure are exercised. Error handling paths:

paths that handle errors are exercised.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 19

Test case: Test data; should be coupled with expected results. Often lists Test identifier; Test objectives; Test

resources; Test procedure. Driver:

A module that has the test case data, passes it to component under testing, and logs the results.

Stub: A dummy module that is called by the component

under testing.

Unit Test: Test Setup

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 20

Example (Stub)

Module A

Stub B Stub C Stub D

Stub B

Print “in B”Return

Module A

Call Stub BCall Stub CCall Stub D

Stub C

Print “in C”Return

Stub D

Print “in D”Return

Temporary Implementation

Module Under Test

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 21

Generating Unit Test Cases

Statement coverage. Graph based:

Branch coverage. Condition coverage. Path coverage.

These are also applicable to testing methods within a class.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 22

Unit Testing: Object Oriented Code Class level testing:

Operations in the class are smallest testable units.

Testing single operation in isolation is difficult. Unit test is generally driven by structure of

methods of the class and the state behavior of class.

Recall - the state-transition model and the state of an object.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 23

Unit Testing: Object Oriented Code A public method in a class can be tested using a

black-box approach. Start from the specification - class interface.

Consider each parameter in the method signature, and identify its possible data range(s): Equivalence Classes - a black box test approach.

Incorporate pre-conditions and post-conditions in the test of a method.

Test exceptions. For complex logic, also use white-box testing or static

testing.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 24

Unit Testing: Object Oriented Code For private methods,

either modify the class (temporarily) so that it can be tested externally: change the access to public.

or incorporate a test driver within the class. or use static test methods, such as program

tracing.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 25

Using Statechart

Create test cases corresponding to each transition path that represent a full object life cycle.

Make sure each transition is exercised at least once.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 26

Example (Account from Lecture 10)

statechart

deposit [amt < -bal]/bal += amt

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 27

Example (cont)

Possible Test Cases: init, deposit(500), withdraw(500):

Should be in InCredit state. init, deposit(10), withdraw(20), deposit(5):

Should be in OverDrawn state. init, deposit(10), withDraw(20), deposit(50),

suspend(), unsuspend(): Should be in InCredit state.

The above test cases exercised each transition at least once.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 28

Test Stage: Integration Testing The entire system is viewed as a collection of

subsystems (sets of classes) determined during the system and object design.

The order in which the subsystems are selected for testing and integration determines the testing strategy.

It is carried out by the Development Engineer. A test that leads to construction of the complete

software architecture.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 29

Integration Testing Big bang integration:

all components together. Bottom up integration:

from lower levels. no test stubs needed.

Top down integration: from higher levels. no test drivers needed.

Sandwich testing: combination of bottom-up and top-down. top layer with stubs and bottom layer with drivers.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 30

Example

A

B C D

GFE

Layer I

Layer II

Layer III

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 31

General Observations Low level components usually perform more

common tasks: Input/Output. Repetitive calculations. Object Oriented: usually entity classes.

High level components usually direct the system activities: Controller.

Summary on functionality: Low Level : More specific (Utility Modules). High Level: More general (Controller Modules).

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 32

Big-Bang Integration

Unit Test F

Unit Test E

Unit Test D

Unit Test C

Unit Test B

Unit Test A

System Test

Unit Test G

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 33

Bottom-Up Integration

A

B C D

GFE

Layer I

Layer II

Layer III

Test F

Test E

Test G

Test C

Test D,G

Test B, E, F

Test A, B, C, D,

E, F, G

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 34

Top-Down Integration

A

B C D

GFE

Layer I

Layer II

Layer III

Test A

Layer I

Test A, B, C, D

Layer I + II

Test A, B, C, D,

E, F, G

All Layers

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 35

Sandwich Integration

A

B C D

GFE

Layer I

Layer II

Layer III

Test F

Test ETest B

Test G

Test D

Test A

Test B, E, F

Test D,G

Test A,C

Test A, B, C, D,

E, F, G

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 36

Sandwich Integration

Combination of Top-Down and Bottom-Up approaches.

Select a target level (e.g., Layer II in slide 35): For level above the target level, perform Top-Down

integration. For level below the target level, perform Bottom-Up

Integration. Exercise both Controller modules and Utility

modules at the same time.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 37

Integration Test: Object Oriented Code Class Clusters:

Classes that are tightly coupled are good candidates for an increment integration.

Candidate class clusters: Classes in a package. Classes in a class hierarchy. Classes associated with the interaction diagram for a

use case. Use based testing:

group the classes – independent / dependent. Begin by testing independent classes. Then, test dependent classes which use independent classes.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 38

Class Cluster: Using Interaction Diagram

:Controller :X :Y:UI

request(r)methodX()

methodY()

reportMethod to test: request ( )Input: rExpected output: a report with specific information.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 39

Class Cluster: Using Use Cases

ATM:

Use Cases: Withdraw Money. Deposit funds. Transfer funds. Change PIN.

Actor: Bank customer.

WithdrawMoney

Bank Customer

DepositFunds

TransferFunds

Change PIN

Use Case Diagram

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 40

Summary for Object Oriented Testing

Use CaseAnalysis

Method Testing

Class Design

Method Design

Method Coding

Class Testing

Cluster Testing

Use Case Testing

Unit Testing

Integration Testing

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 41

Test Stage: System Testing

Performed exclusively in terms of inputs and outputs of the system.

Performed mostly on the target platform. Thread-based:

The behavior that results from a system level input. An interleaved sequence of system inputs (stimuli)

and outputs (responses). Depicts a possible scenario of using the system.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 42

Test Stage: Testing that Involves Users Alpha testing:

In-house testing. By a test team or end users.

Beta testing: By users or selected subset of actual customers in a

normal work environment. Product is very close to completion. Open beta release: Let public carry out the beta testing.

Acceptance testing: By users to check that system satisfies requirements to

decide whether to accept the system based on the test result.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 43

Black box versus White box approach: Black box testing:

Knowing the specified function a component has been designed for.

Tests conducted at the interface of the component.

White box testing: Knowing the internal workings of a component. Test cases exercise specific sets of condition,

loops, etc.

Test Case Design: Overview

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 44

Another way to classify the test cases: Test to specifications:

Also known as black-box, data-driven, functional, or input/output driven testing.

Ignore the code — use the specifications to select test cases. Test to code:

Also known as glass-box, logic-driven, structured, or path-oriented testing.

Ignore the specifications — use the code to select test cases. Neither exhaustive testing to specifications nor

exhaustive testing to code is feasible.

Test Case Design: Overview

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 45

Project: Test Phase

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 46

Requirements for Test Phase

Automated Test Driver: Read test cases from a file. Put them through the system. Capture and validate output.

Test Case Documentation: Based on Use Cases (Black Box). Three Tests per Use Case:

One Valid, Two Invalid. Validity should be based on functionality.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 47

Automated Test Driver In essence, an Automated Test Driver replaces

the Boundary Classes in your system: Read directly from a file instead of reading input from

an actual user. Get output directly from the system and place them

into a file instead of showing on screen.

AutomatedTest

Driver

Input File

Output File

ApplicationLayer

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 48

Automated Test Driver: Input File The input file:

Text base: Facilitate modification of test cases. Contains a number of test cases.

Each of the test cases contains: The method to be tested. Parameter(s) for the method. The expected output. Comments (optional).

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 49

Automated Test Driver: Output File The output file:

Text base: Easy for human inspection. Test case result.

Each test case result contains: Output of method (if any). Common Categories of Test Result:

Pass: Test result == Expected result. Failed: Test result != Expected result. Human Check: For more complicated cases where the

expected result is hard/impossible to be defined.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 50

Test Case Design

The Automated Test Driver can be used to test in different test phases: Unit Testing. Integration Testing. Etc.

For your project, test cases should be derived from Use Cases.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 51

Example (Use Case Diagram) A very simple system to calculate and provide

comments for students’ CAP (Cumulative Average Percentile on a scale 1-5) points:

User

Calculate CAP

Give Comment

CAP System

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 52

Example (Use Case Descriptions)Calculate CAP: Basic Course of Events

User enters a number of grades. System displays CAP.

Give Comment: Basic Course of Events

User enters CAP. System displays Comments based on CAP:

If CAP > 4.0 → "Excellent“; else if CAP >3.0 → "Good“; else if CAP >2.0 → "Work hard"; otherwise, → "Work extremely hard“.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 53

Example (Class Diagram)

CAP

computeCAP(grades: double[*])

comment(CAP: double): String

etc...

CAP_UI

displayMenu( )

etc...

mySystem

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 54

Example (Class Design)

Automated Test Driver takes the place of CAP_UI class:

Two Classes: CAPTD: main body of the test driver:

A simple loop to go through all test cases. Makes use of TDIO for reading and writing.

TDIO: utility class: Handles input/output. Keeps track of pass/fail/check cases. Contains some useful methods for the test driver in general. Can be reused for your own Test Driver.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 55

Example (Class Design)

The TDIO class:

TDIO

TDIO(ifile: String, ofile: String)

getInput(delim: String): String[*]

pass()

fail(s: String)

check(s: String)

endTest()

Open “ifile” for reading.Open “ofile” for writing.

Read a single line from input file, tokenized using delim as delimiter.

Write the test result to output file. Keep track of the number of each cases.

Write the accumulated statistics to a file.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 56

Example (Input File)

// The format of the test data for each // method is shown first.// comment,cap[,comment-expected]comment,4.2,Excellentcomment,1,too badcomment,3.5comment,5.0,too good to be true// computeCAP,grade points separated by comma,// cap-expectedcomputeCAP,1,2,3,4,5,3// End of test data file

The test cases (input file):

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 57

Example (Test Driver)

public static void main (String[] args) throws IOException { TDIO t = new TDIO("CAPTDin.txt", "CAPTDout.txt"); CAP c = new CAP(); while ((data = t.getInput(",")) != null) { if (data[0].equals ("comment")) { returnedString = c.comment(Double.parseDouble(data[1])); if (data.length == 3) if (returnedString.equals(data[2])) t.pass(); else t.fail( returnedString ); else t.check( returnedString ); else if (data[0].equals ("computeCAP")) } ...... } t.endTest();}

The CAPTD class (just a main() method):

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 58

Example (Output File)

comment,4.2,Excellent// passcomment,1,too bad// FAIL! Result expected -->Work extremely hardcomment,3.5// Check result! -->Good

... OMITTED ...comment,5.0,too good to be true// FAIL! Result expected -->ExcellentcomputeCAP,1,2,3,4,5,3// passpass = 2fail = 2Manual Check = 1total (pass+fail) = 4Passing rate (pass/total) = 0.5

The output file:

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 59

Example (Wrapping Up)

Test Case Documentation can be done directly in the input file.

A good test driver coupled with good test cases will significantly reduce the demonstration time.

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 60

Where are we now?

Requirement

Analysis

Design

Implement

Test

Evaluating the System

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 61

Summary

Software Testing Overview Test Phases

Unit Testing Integration Testing

OO Specific Testing Automated Test Driver

Overview Example

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 62

Reading Suggestions

Chapter 9 of [Bimlesh, Andrei, Soo; 2007] Chapter 23 of [Somerville; 2007] - Testing Chapter 9 of [Pfleeger, Atlee; 2006]

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 63

Coming up next

Chapter 9 of [Bimlesh, Andrei, Soo; 2007] Chapter 23 of [Somerville; 2007] - Testing Chapters 8 and 9 of [Pfleeger, Atlee; 2006]

04/18/23CPSC-4360-01, CPSC-5360-01,

Lecture 11 64

Thank you for your attention!

Questions?