PROGRAMMING TESTING 201300071-1B MODULE 2: SOFTWARE SYSTEMS 22 NOVEMBER 2013.

PROGRAMMING

TESTING201300071-1B MODULE 2: SOFTWARE SYSTEMS

22 NOVEMBER 2013

Week 1• Values, conditions• Classes, objects

Week 2• Specifications• Testing

Week 3• Abstraction,

inheritance

Week 4• Interfaces,

abstract classes• Arrays

Week 5• Collections• Generic structures

Week 6• Exceptions• Stream I/O• GUIs

Week 7• Concurrency• Networking

Week 8• Security

Week 9/10Project

Software Systems - Programming 2

OVERVIEW PROGRAMMING LINE

System testing & unit testing Test plan Test framework Test coverage

Nino & Hosch: Chapter 6Tool support: Emma (EclEmma for Eclipse)Self study: Nino & Hosch: Chapters 7 & 8


CONTENTS

Software Systems - Programming

System testing (functional testing) Complete system, final configuration Test system behavior, e.g., use cases Typically performed by testing team

Unit testing Class, method, etc. in isolation Test unit’s behavior Typically performed by developer of unit

Further tests: Performance , Stress , Platform , Usability∼ ∼ ∼ ∼ Out of scope for this course

4

OVERVIEW: TESTING TERMINOLOGY

Test whether the system or unit Behaves as intended When used in the specified way

Plan: List of execution scenarios Describe how to perform scenario Describe expected outcome


TEST PLAN

Use cases as execution scenarios, consider Alternative flows in use cases Different values used in use cases Additional requirements

Treat system as “black box” Internal structure is unknown Validation by observing behavior


TEST PLAN: SYSTEM TESTING

One test plan per class or method, can also be Per group of logically related methods Per group of classes in the same hierarchy

Consider method contracts, test that The postcondition of methods holds When the precondition is satisfied

Treat method as “white box” Internal structure is known E.g., consider conditionals Validation: by inspecting method result and/or instance/class variables


TEST PLAN: UNIT TESTING

Lecture will further focus on unit testing.

For the project, you need to perform both, system

and unit tests!

Identify relevant cases

Example: Lock Four cases

Attempt to open in open state Attempt to open in closed state Attempt to close in open state Attempt to close in closed state

Actually many more situations: Do we have to test for all possible values? 1000 possibilities for lockCode 1000 possibilities for code passed to enterDigits

→ isOpen()→ isOpen() <==> code correct→ not isOpen()→ not isOpen()


TEST CASES

Lock-int lockCode+boolean isOpen()+void close()+void enterDigits(int code)

> 2,000,000combinations

Do we have to test for all possible values? NO! Pick representative values

What is representative? Group values for which the unit under test behaves the same Pick samples from this group

Random value Boundaries (e.g,. maximum and minimum)

E.g,. for the Lock: pick 0, 123, 999


TEST CASES

Developers are like to make mistakesat boundaries

So called off-by-one errors

Example: ModuloCounter Specification

Count from 0 up to 9 When increasing with a value of 9,

reset to 0 instead Test

Invoke increase: zero times, five times, nine times, ten times


BOUNDARY CASES

ModuloCounter-int value+int getValue()+void increase()

this.value = this.value + 1;if (value >= 9) { this.value = 0:}

off-by-one error

Primarily test for correct behaviour when input is within specified bounds

Sometimes we want to test that nothing bad happens when input is wrong E.g., data entered through the user interface Test robustness of software

Can only be tested if the system behavior for wrong input is specified E.g., usage of default values instead of wrong input

Beware: documenting behaviour outside specified precondition encourages to ignore preconditions! Specification of exceptional behaviour: see week 6


TESTING ILLEGAL DATA

One class per test plan Implement each test case as method Test method uses unit under test and asserts correct behaviour

Compare actual outcome with expected outcome Often test cases share preparatory tasks, e.g., creating instance of class

under test Extract shared code into setUp() method setUp() method executed before each test method Establishes defined initial situation


COMMON STRUCTURE OF UNIT TEST

JUnit (www.junit.org) Provides framework for writing and running unit tests Uses Java features not introduced in this course

Provided: simplified version of Junit Main class (Test) Test class per class-under-test

Main class is executable (has main method) Instantiates and test classes and invokes their method runTest()

Test classes implement test plan Method runTest() must: invoke each test method invoke setUp() before each test method


TEST FRAMEWORK

http://www.junit.org/


If my code passes all tests, I’m done!

Is this correct?

How do I assure a good quality of my tests? Systematically develop tests based on contracts Review tests Measure coverage Other techniques, not covered in this course

QUALITY OF TESTS

Statement Each statement executed at least once

Branch Each branch traversed (and every entry point taken) at least

once

Path All program paths traversed at least once


TYPES OF COVERAGE

100% coverage difficult to achieve Especially for branch or path coverage, too many test cases would be

needed Synthetic code generated by compiler Dead code (should be eliminated when discovered) Code handling exceptional situations

High statement coverage generally possible (e.g., > 80%)


GOALS FOR CODE COVERAGE

Open-source tool Supports class, method, and line coverage.

“Fractional” line coverage supported, but not branch coverage. Eclipse plugin EclEmma also available

http://www.eclemma.org


EMMA


EMMA COVERAGE REPORT


EMMA SOURCE CODE ANNOTATIONS


FRACTIONAL LINE COVERAGE

Only partof conditionalexecuted

Loop incrementnot executed

Java compiler may emit multiple copies of code, e.g., for instance field initializer Emma expects

coverage of all copies


ANOMALIES IN REPORTED COVERAGE

Java compiler may generate default code, e.g., for missing constructors Emma also reports

coverage of such code

Iterative process Perform unit tests always after functionality is complete Test from the beginning of the implementation Extending implementation requires extending test plan Test results lead to further coding Test coverage results lead to extending test plan

Philosophies “Code a little, test a little” “Test a little, code a little” (test-driven development, test first)


TESTING / DEVELOPMENT PROCESS

Test plan System tests based on use cases Unit tests based on method contracts

Test representatives and boundary cases Confirm quality of your tests by measuring their code coverage


MAIN POINTS

PROGRAMMING TESTING 201300071-1B MODULE 2: SOFTWARE SYSTEMS 22 NOVEMBER 2013.

Documents

Transcript of PROGRAMMING TESTING 201300071-1B MODULE 2: SOFTWARE SYSTEMS 22 NOVEMBER 2013.