Test-Driven Design - SE767: Vérification & Test · Test-Driven Design SE767: Vériﬁcation & Test...

Test-Driven DesignSE767: Vérification & Test

Ulrich Kü[email protected]/11/2018

Plan

Test-Driven Development

TDD for Embedded C with unity

2/28 Ulrich Kühne 27/11/2018

Motivation

Why should I write the tests first??

Time

Mistakemade

Bugdiscovered

Bugfound

Bugfixed

Tdisc Tfind Tfix

TimeMistakemade Bug

discovered

Bugfound Bug

fixed

Tdisc Tfind Tfix

3/28 Ulrich Kühne 27/11/2018

Advantages of TDD

Improved software quality (fewer bugs)Reduced debugging timeLeads to clean and modular designAllows for safe code changesExecutable documentationKeeps track of progressPredictable development time

4/28 Ulrich Kühne 27/11/2018

TDD Cycle

1. Write a test2. Run the tests and watch it fail3. Write the code to make it pass4. Run the tests and see it works5. Refactor code

5/28 Ulrich Kühne 27/11/2018

Getting Started

Which tests should I write??

Writing a test plan

What behavior should be implemented?What output do we expect?Input constraintsFailure conditions

6/28 Ulrich Kühne 27/11/2018

Example 1: Circular buffer (first in first out)

Requirements

R1 After creation, the buffer is emptyR2 The buffer can hold up to 32 integer numbers (elements)R3 The user can query if the buffer is emptyR4 The user can query if the buffer is fullR5 The user can query the number of elements in the bufferR6 The buffer is empty iff it holds zero elementsR7 The buffer is full iff it holds 32 elementsR8 The user can insert elements into the queue iff it is not fullR9 The user can retrieve the oldest element in the queue

Write a list of tests for the circular buffer!!

7/28 Ulrich Kühne 27/11/2018

Plan

Test-Driven Development

TDD for Embedded C with unity

8/28 Ulrich Kühne 27/11/2018

The Unit Testing Framework unity

Why do we need a testing framework??

Build automation and integrationTest automationLow coding overheadConvenient assertion macros

9/28 Ulrich Kühne 27/11/2018

Unit Test Harness

A test harness provides. . .a common way to express testcases,a common way to expressexpected results,a place to collect test cases for aproject, system, or subsystem,a mechanism to run the testcases, anda report of the test results.

10/28 Ulrich Kühne 27/11/2018

unity Test Harness

Let’s write some tests. . .

11/28 Ulrich Kühne 27/11/2018

Some Fundamental Rules for TDD

1. Write the tests first!2. Watch the tests fail!3. Keep your tests short and concise!4. Write the minimal code to make tests pass!

12/28 Ulrich Kühne 27/11/2018

Hardware Dependencies

What to do if the hardware is not available??

Fake it!

13/28 Ulrich Kühne 27/11/2018

Test Doubles

Test dummy Does nothing, makes the sources compile.Test stub Returns some (constant) value.Test spy Captures parameters to allow sanity checks.

Mock object Verifies functions called, call order, parameters.Fake object Simplified implementation.

Simulator Software-based implementation, can be ascomplex as the real thing.

14/28 Ulrich Kühne 27/11/2018

Example 2: LED Driver

Requirements

R1 The LED driver controls 8 LEDs.R2 The driver allows to turn on or off any of the LEDs individually.R3 The driver allows to turn on or off all LEDs with a single call.R4 The user can query the state of any LED.R5 The LEDs are memory-mapped to an 8-bit register.R6 After creation of the driver, all LEDs are off.R7 The LEDs are addressed ranging from index 1 (least significant

bit) to 8 (most significant bit).

Write a list of tests for the LED driver!!

15/28 Ulrich Kühne 27/11/2018

Example 2 (continued)

Let’s write some tests. . .

16/28 Ulrich Kühne 27/11/2018

Breaking (Hardware) Dependencies

Test

Codeunder test

Module1

Module2

HardwareA

HardwareB RTOS

HardwareC

Module3

HardwareD

17/28 Ulrich Kühne 27/11/2018

Breaking (Hardware) Dependencies

Test

Codeunder test

TestDouble

TestDouble

HardwareA

HardwareB RTOS

HardwareC

Module3

HardwareD

17/28 Ulrich Kühne 27/11/2018

Advantages of Using Test Doubles

Breaking complex (hardware) dependenciesEnables testing ahead of available hardwareLeads to clear interfaces and abstractions (modular code)Enables testing rare conditions and failuresSpeeds up build and test processServes as documentation for replaced (hardware) modules

18/28 Ulrich Kühne 27/11/2018

Example 3: Light Scheduler

Description

Home automationSwitching of lightsSchedule events (on/off)Everyday, specific day, orweekendRandomize switching timeRTOS wakeup callback

Dependencies

Lightscheduler

Lightcontroller

Timeservice

Hardware RTOS

19/28 Ulrich Kühne 27/11/2018

Example Use Casemain LightScheduler LightControl TimeService

scheduleOn(2,SUNDAY,1200)

20/28 Ulrich Kühne 27/11/2018



Sunday, 8:00

20/28 Ulrich Kühne 27/11/2018



Sunday, 8:00wakeUp()

20/28 Ulrich Kühne 27/11/2018



Sunday, 8:00wakeUp()

turnOn(2)

20/28 Ulrich Kühne 27/11/2018



Sunday, 8:00

Sunday, 8:01

wakeUp()

turnOn(2)

wakeUp()

20/28 Ulrich Kühne 27/11/2018

Test Infrastructure

How could we possibly test this thing??

Breaking dependencies using test doublesLightControl

• Replace it by a test spy• Check status of (fake) lights in test bench

TimeService

• Replace it by a fake object• Set current time in test bench• Trigger wakeUp() callback from test bench

21/28 Ulrich Kühne 27/11/2018

Example 3 (Continued)

⇒ Exercise!

22/28 Ulrich Kühne 27/11/2018

Mock Objects

Testing of complex(hardware) interactionsTesting protocolcomplianceSometimes order mattersTesting rare behaviorTesting failure conditions

Application

Driver

Hardware

23/28 Ulrich Kühne 27/11/2018

Mock Objects

Testing of complex(hardware) interactionsTesting protocolcomplianceSometimes order mattersTesting rare behaviorTesting failure conditions

Application

Driver

Hardware

Test

Driver

Mock object

23/28 Ulrich Kühne 27/11/2018

Example: Flash Driver

b7 == 1

Start

Program CommandWrite 0x40 to 0x0

Start/Stop

Write data toaddress

Read statusregister

b3 == 0

b4 == 0

b1 == 0

YES

NO

YES

YES

YES

Vpp Error

Program Error

Protected BlockError

NO

NO

NO

Clear statusWrite 0xFF to 0x0

Wai

t for

read

y lo

op

Figure: Flash memory program flowchart (from [1])

24/28 Ulrich Kühne 27/11/2018

Testing with Mock Objects

1. Test code tells mock objectexpectations on its use

2. Code under test is executed3. Mock object records and checks calls

via its interface4. Test code verifies that all expected calls

have completed

Test

Driver

Mock object

25/28 Ulrich Kühne 27/11/2018

Example Test Code01: uint16_t address = 0x1000;

02: uint16_t data = 0xCAFE;

03:

04: void TestWriteSucceeds(void)

05: {

06: MockFlash_Expect_Write(0x0, 0x40);

07: MockFlash_Expect_Write(address, data);

08: MockFlash_Expect_ReadThenReturn(0x0, 1<<7);

09: MockFlash_Expect_Write(0x0, 0xFF);

10:

11: TEST_ASSERT_EQUAL_INT(0, Flash_Write(address, data));

12:

13: MockFlash_Verify_Complete();

14: }

26/28 Ulrich Kühne 27/11/2018

Refactoring

Closes the TDD cyclePrepares next TDD cycleCleaning up without breaking thetests

• Add comments where needed• Move common functionality to a

separate function• Break up complex conditions• . . .

27/28 Ulrich Kühne 27/11/2018

Summary

Advantages

Structured methodology forembedded software designPredictable progressNo untested codeShort debugging cyclesLeads to modular designBreak hardwaredependencies

Disadvantages

Real-time systems difficultto testInconsistency betweenhardware and test doublesRequires disciplineLong integration of legacycode

28/28 Ulrich Kühne 27/11/2018

References I

James W. Grenning.Test Driven Development for Embedded C.Pragmatic Bookshelf, Raleigh, N.C, 1st edition, May 2011.

29/28 Ulrich Kühne 27/11/2018

Test-Driven Design - SE767: Vérification & Test · Test-Driven Design SE767: Vériﬁcation & Test...

Documents

Transcript of Test-Driven Design - SE767: Vérification & Test · Test-Driven Design SE767: Vériﬁcation & Test...