Test & Verification Verification Kim G. Larsen. TOV 2002Kim G. Larsen 2 Research Profile Distributed...

TestTest && VerificationVerification

Kim G. Larsen

2TOV 2002 Kim G. Larsen

Research ProfileDistributed Systems & Semantics Unit

Semantic Models concurrency, mobility, objects real-time, hybrid systems

Validation & Verificationalgorithms & tools

Construction real-time & network systems


BRICS Machine Basic Research in Computer Science

30+40+40 Millkr

100

100

Aalborg Aarhus

ToolsOther revelvant projects UPPAAL, VHS, VVS, WOODDES


Tools and BRICS

Logic• Temporal Logic• Modal Logic• MSOL • •

Algorithmic• (Timed) Automata Theory• Graph Theory• BDDs• Polyhedra Manipulation• •

Semantics• Concurrency Theory• Abstract Interpretation• Compositionality• Models for real-time & hybrid systems• •

HOL TLP

Applications

PVS ALF

SPINvisualSTATE UPPAAL


A very complex system

Klaus Havelund, NASA


Rotterdam Storm Surge Barrier


Spectacular Bugs

ARIANE-5 INTEL Pentium II floating-point division

470 Mill US $ Baggage handling system, Denver

1.1 Mill US $/day for 9 months Mars Pathfinder Radiation theraphy, Therac-25 ……. More in JPK, CW


Embedded Systems

SyncMaster 17GLsi

Telephone

Tamagotchi

Mobile Phone

Digital Watch


A simple program

Int x

Process INCdo:: x<200 --> x:=x+1od

Process DECdo:: x>0 --> x:=x-1od

Process RESETdo:: x=200 --> x:=0od

fork INC; fork DEC; fork RESET

Int x

Process INCdo:: x<200 --> x:=x+1od

Process DECdo:: x>0 --> x:=x-1od

Process RESETdo:: x=200 --> x:=0od

fork INC; fork DEC; fork RESET

Which values mayx take ?

Questions/Properties:E<>(x>1000)E<>(x>2000)A[](x<=2000)E<>(x<0)A[](x>=0)Possibly

Always


Introducing, Detecting and Repairing Errors Liggesmeyer 98


Suggested Solution?

Model based validation, verfication and testing

of software and hardware


Verification & Validation

Design Model Specification

Analysis

Implementation

Testing



Design Model SpecificationVerification & Refusal

AnalysisValidation

Implementation

Testing

UML

SDL




AnalysisValidation

Implementation

Testing

UML

SDL

ModelExtraction

AutomaticCode generation




AnalysisValidation

Implementation

Testing

UML

AutomaticCode generation

AutomaticTest generation

SDL

ModelExtraction


How?

Unified Model = State Machine!

a

b

x

ya?

b?

x!

y!b?

Control states

Inputports

Outputports


TamagotchiA C

Health=0 or Age=2.000

B

Passive Feeding Light

Clean

PlayDisciplineMedicine

Care

Tick

Health:=Health-1; Age:=Age+1

AA

A

A

AA

A

A

Meal

Snack

B

B

ALIVE

DEAD

Health:= Health-1


SYNCmaster


Digital Watch


The SDL EditorThe SDL EditorThe SDL Editor

Process levelProcess level


SP

IN, G

erald H

olzm

ann

AT

&T


visualSTATE

Hierarchical state systems

Flat state systems Multiple and inter-

related state machines

Supports UML notation

Device driver access

VVS w Baan Visualstate, DTU (CIT project)


ESTEREL


UP

PA

AL


‘State Explosion’ problem

a

cb

1 2

43

1,a 4,a

3,a 4,a

1,b 2,b

3,b 4,b

1,c 2,c

3,c 4,c

All combinations = exponential in no. of components

M1 M2

M1 x M2

Provably theoretical

intractable


Train Simulator1421 machines11102 transitions2981 inputs2667 outputs3204 local statesDeclare state sp.: 10^476

BUGS ?

VVSvisualSTATE

Our techniuqes has reduced verific

ation

time w

ith several orders of magnitude

(ex 14 days to 6 sec)


Tool Support (model checking)

System Description A

Requirement FYes, Prototypes Executable Code Test sequences

No!Debugging Information

Tools: UPPAAL, visualSTATE, ESTEREL, SPIN, Statemate, FormalCheck, VeriSoft, Java Pathfinder,…

TOOLTOOL

VVS

Verification and Validation of Large Systems

DTU, Aalborg,Baan Visualstate

URLs://www.visualSTATE.com //www.it.dtu.dk/~jst/vvs/


BAAN VisualSTATE Tidligere BEOLOGIC

Beologic’s Products: salesPLUSsalesPLUS visualSTATEvisualSTATE

1980-95: Independent division of B&01995- : Independent company

B&O, 2M Invest, Danish Municipal Pension Ins. Fund

Customers:ABBB&ODaimler-BenzEricson DIAXESA/ESTECFORDGrundfosLEGOPBSSiemens ……. (approx. 90)

Verification Problems:• 1.000 components• 10400 states

Our techniques has reducedverification by an order of magnitude

(from 14 days to 6 sec)

•Embedded Systems•Simple Model•Verification of Std. Checks•Explicit Representation (STATEEXPLOSION)•Code Generation


visualSTATE 4.0 Product Modules

NavigatorPrototyper

Graphical Simulation of human interface panels

Presenter Prototyper for

distribution

Designer Diagram Designer Matrix Designer Text Editor

Tester Validator

SimulationAnimationAnalysis

VerificatorStatic verificationDynamic verification

Generator Coder Documentor


visualSTATE Prototyper

A virtual prototype ofa mobile telephone

GUI BuilderGUI ExecuterPick’n place of symbolsNo manual codingCustom designed

objects ActiveX controls Graphics libraries


visualSTATE Designer

Hierarchical state systems

Flat state systems Multiple and inter-

related state machines

Supports UML notation

Device driver access


No local nor global dead-ends No never interpreted events No fired actions No conflicting transactions No unreachable states

All combinations are checked!

visualSTATE Tester Verification

100%Tested!

No bugs allowed!


Train Simulator1421 machines11102 transitions2981 inputs2667 outputs3204 local statesDeclare state sp.: 10^476

BUGS ?

VVS


Experimental Breakthroughs

State Space St-of-Art ComBackSystem Mach.Declared Reach

Checks VisualST Sec MB Sec MB

VCR 7 10 5̂ 1279 50 <1 <1 6 <1 7JVC 8 10 4̂ 352 22 <1 <1 6 <1 6HI-FI 9 10 7̂ 1416384 120 1200 1.0 6 3.9 6Motor 12 10 7̂ 34560 123 32 <1 6 2,0AVS 12 10 7̂ 1438416 173 3780 6.7 6 5.7 6Video 13 10 8̂ 1219440 122 --- 1.1 6 1.5 6Car 20 10 1̂1 9.2 10 9̂ 83 --- 3.8 9 1.8 6N6 14 10 1̂0 6399552 443 --- 32.3 7 218 6N5 25 10 1̂2 5.0 10 1̂0 269 --- 56.2 7 9.1 6N4 23 10 1̂3 3.7 10 8̂ 132 --- 622 7 6.3 6Train1 373 10^136 --- 1335 --- --- --- 25.9 6Train2 1421 10^476 --- 4708 --- --- --- 739 11

Machine: 166 MHz Pentium PC with 32 MB RAM

---: Out of memory, or did not terminate after 3 hours.

VVS project BRICS/Aalborg, DTU, BAAN visualSTATE


Experimental BreakthroughsPatented

State Space St-of-Art ComBackSystem Mach.Declared Reach

Checks VisualST Sec MB Sec MB

VCR 7 10 5̂ 1279 50 <1 <1 6 <1 7JVC 8 10 4̂ 352 22 <1 <1 6 <1 6HI-FI 9 10 7̂ 1416384 120 1200 1.0 6 3.9 6Motor 12 10 7̂ 34560 123 32 <1 6 2,0AVS 12 10 7̂ 1438416 173 3780 6.7 6 5.7 6Video 13 10 8̂ 1219440 122 --- 1.1 6 1.5 6Car 20 10 1̂1 9.2 10 9̂ 83 --- 3.8 9 1.8 6N6 14 10 1̂0 6399552 443 --- 32.3 7 218 6N5 25 10 1̂2 5.0 10 1̂0 269 --- 56.2 7 9.1 6N4 23 10 1̂3 3.7 10 8̂ 132 --- 622 7 6.3 6Train1 373 10^136 --- 1335 --- --- --- 25.9 6Train2 1421 10^476 --- 4708 --- --- --- 739 11

Machine: 166 MHz Pentium PC with 32 MB RAM

---: Out of memory, or did not terminate after 3 hours.

Our techniques h

ave reduced

verification tim

e with

several orders of m

agnitude

(ex fro

m 14 days to 6 se

c)

UPPAALUPPAAL

Modelling and Verification of Real Time systems

UPPAAL2k > 800 users > 35 countries

UPPAAL2k > 800 users > 35 countries


Collaborators@UPPsala

Wang Yi Johan Bengtsson Paul Pettersson Fredrik Larsson Alexandre David Tobias Amnell Oliver Möller

@AALborg Kim G Larsen Arne Skou Paul Pettersson Carsten Weise Kåre J Kristoffersen Gerd Behrman Thomas Hune Oliver Möller Nicky Oliver Bodentien Lasse Poulsen

@Elsewhere David Griffioen, Ansgar Fehnker, Frits Vandraager, Klaus Havelund, Theo

Ruys, Pedro D’Argenio, J-P Katoen, J. Tretmans,Judi Romijn, Ed Brinksma, Franck Cassez, Magnus Lindahl, Francois Laroussinie, Patricia Bouyer, Augusto Burgueno, H. Bowmann, D. Latella, M. Massink, G. Faconti, Kristina Lundqvist, Lars Asplund, Justin Pearson...


Dec’96 Sep’98


Dec’96 Sep’98

from7.5 hrs / 527 MB on ONYX with 2GB (4Mill DKK)to12.75 sec / 2.1 MB on Pentium 150 MHz, 32 MBorEvery 9 month 10 times better performance!


Hybrid & Real Time Systems

PlantContinuous

Controller ProgramDiscrete

Control Theory Computer Science

Eg.:Pump ControlAir BagsRobotsCruise ControlABSCD PlayersProduction Lines

Real Time SystemA system where correctness not only depends on the logical order of events but also on their timing

Real Time SystemA system where correctness not only depends on the logical order of events but also on their timing

sensors

actuators

TaskTask

TaskTask


Construction of UPPAAL models

PlantContinuous

Controller ProgramDiscrete

sensors

actuators

TaskTask

TaskTask

a

cb

1 2

43

a

cb

1 2

43

1 2

43

1 2

43

a

cb

UPPAAL Model

Modelofenvironment(user-supplied)

Model oftasks(automatic?)


Intelligent Light Control

Off Light Brightpress? press?

press?

press?

WANT: if press is issued twice quickly then the light will get brighter; otherwise the light is turned off.


Intelligent Light Control

Off Light Brightpress? press?

press?

press?

Solution: Add real-valued clock x

X:=0X<=3

X>3


Timed Automata

n

m

a

Alur & Dill 1990

Clocks: x, y

x<=5 & y>3

x := 0

Guard Boolean combination of integer boundson clocks and clock-differences.

ResetAction perfomed on clocks

Transitions

( n , x=2.4 , y=3.1415 ) ( n , x=3.5 , y=4.2415 )

e(1.1)

( n , x=2.4 , y=3.1415 ) ( m , x=0 , y=3.1415 )

a

State ( location , x=v , y=u ) where v,u are in R

Actionused

for synchronization


n

m

a

Clocks: x, y

x<=5 & y>3

x := 0

Transitions

( n , x=2.4 , y=3.1415 ) ( n , x=3.5 , y=4.2415 )

e(1.1)

( n , x=2.4 , y=3.1415 )

e(3.2)

x<=5

y<=10

LocationInvariants

g1g2 g3

g4

Timed Automata Invariants

Invariants ensure

progress!!

Invariants ensure

progress!!


The UPPAAL Model= Networks of Timed Automata + Integer Variables +….

l1

l2

a!

x>=2i==3

x := 0i:=i+4

m1

m2

a?

y<=4

………….Two-way synchronizationon complementary actions.

Closed Systems!

Two-way synchronizationon complementary actions.

Closed Systems!

(l1, m1,………, x=2, y=3.5, i=3,…..) (l2,m2,……..,x=0, y=3.5, i=7,…..)

(l1,m1,………,x=2.2, y=3.7, I=3,…..)

0.2

tau

Example transitions

If a URGENT CHANNEL


LEGO Mindstorms/RCX

Sensors: temperature, light, rotation, pressure.

Actuators: motors, lamps,

Virtual machine: 10 tasks, 4 timers,

16 integers.Several Programming Languages:

NotQuiteC, Mindstorm, Robotics, legOS, etc.

3 input ports

3 output ports

1 infra-red port


First UPPAAL modelSorting of Lego Boxes

Conveyer Belt

Exercise: Design Controller so that only black boxes are being pushed out

BoxesPiston

Black

red9 18 81 90

99

BlckRd

remove

eject

Controller

Ken Tindell

MAIN PUSH


NQC programs

task PUSH{ while(true){ wait(Timer(1)>DELAY && active==1); active=0; Rev(OUT_C,1); Sleep(8); Fwd(OUT_C,1); Sleep(12); Off(OUT_C); }}

task PUSH{ while(true){ wait(Timer(1)>DELAY && active==1); active=0; Rev(OUT_C,1); Sleep(8); Fwd(OUT_C,1); Sleep(12); Off(OUT_C); }}

int active;int DELAY;int LIGHT_LEVEL;

int active;int DELAY;int LIGHT_LEVEL;

task MAIN{ DELAY=75; LIGHT_LEVEL=35; active=0; Sensor(IN_1, IN_LIGHT); Fwd(OUT_A,1); Display(1);

start PUSH; while(true){ wait(IN_1<=LIGHT_LEVEL); ClearTimer(1); active=1; PlaySound(1); wait(IN_1>LIGHT_LEVEL); }}

task MAIN{ DELAY=75; LIGHT_LEVEL=35; active=0; Sensor(IN_1, IN_LIGHT); Fwd(OUT_A,1); Display(1);

start PUSH; while(true){ wait(IN_1<=LIGHT_LEVEL); ClearTimer(1); active=1; PlaySound(1); wait(IN_1>LIGHT_LEVEL); }}

UPPAAL Demo


From RCX to UPPAAL

Model includes Round-Robin Scheduler.

Compilation of RCX tasks into TA models.

Presented at ECRTS 2000

Task MAIN


The Production CellCourse at DTU, Copenhagen

Production Cell


Train Crossing

River

Crossing

Gate

StopableArea

[10,20]

[7,15]

Queue

[3,5]


Train Crossing

River

Crossing

Gate

StopableArea

[10,20]

[7,15]

Queue

[3,5]appr,stop

leave

go

emptynonemptyhd, add,rem

elel

Communication via channels andshared variable.


Case Studies: Protocols

Philips Audio Protocol [HS’95, CAV’95, RTSS’95, CAV’96]Collision-Avoidance Protocol [SPIN’95]

Bounded Retransmission Protocol [TACAS’97]

Bang & Olufsen Audio/Video Protocol [RTSS’97]

TDMA Protocol [PRFTS’97]

Lip-Synchronization Protocol [FMICS’97]

Multimedia Streams [DSVIS’98]

ATM ABR Protocol [CAV’99]

ABB Fieldbus Protocol [ECRTS’2k]

IEEE 1394 Firewire Root Contention (2000)


Case-Studies: Controllers

Gearbox Controller [TACAS’98]

Bang & Olufsen Power Controller [RTPS’99,FTRTFT’2k]

SIDMAR Steel Production Plant [RTCSA’99, DSVV’2k]

Real-Time RCX Control-Programs [ECRTS’2k]

Experimental Batch Plant (2000)

RCX Production Cell (2000)


Timed Automata in UPPAAL

Timed (Safety) Automata+ urgent actions + urgent locations+ committed locations+ data-variables (with bounded domains)+ arrays of data-variables + constants + guards and assignments over data-variables and arrays…+ templates with local clocks, data-variables, and constants.


Declarations in UPPAAL

clock x1, …, xn;

int i1, …, im;

chan a1, …, ao;

const c1 n1, …, cp np;

Examples:

clock x, y;

int i, J0; int[0,1] k[5];

const delay 5, true 1, false 0;

Array k of five booleans.


Timed Automata in UPPAAL

n

m

a

x<=5 & y>3

x := 0

x<=5

y<=10

g1g2 g3

g4

invinvnxnxinv ,||::

clock natural number and

}!,,,,,{

},,,,{

::

|::

,||::

op

ExpropExprg

nyxnxg

ggggg

d

c

dc

nx :

clock guards

data guards

clock assignments

clock assignments

):?(

|/

|*

|

|

||

|][|::

:

ExprExprg

ExprExpr

ExprExpr

ExprExpr

ExprExpr

Exprn

ExpriiExpr

Expri

d

location invariants


Urgent Channels

urgent chan hurry;

Informal Semantics:• There will be no delay if transition with urgent action can be taken.

Restrictions:• No clock guard allowed on transitions with urgent actions.

• Invariants and data-variable guards are allowed.


Urgent Locations

Click “Urgent” in State Editor.

Informal Semantics:• No delay in urgent location.

Note: the use of urgent locations reduces the number of clocks

in a model, and thus the complexity of the analysis.


Committed Locations

Click “Committed” in State Editor.

Informal Semantics:• No delay in committed location.• Next transition must involve automata in committed location.

Note: the use of committed locations reduces the number of

clocks in a model, and allows for more space and time efficient

analysis.


CSMA/CD protocol – MAC layer

send - service provided by Mac which reacts by transmitting a message, rec - (receive) service provided by Mac, indicates that a message is ready to be received, b - (begin) Mac begins message transmission to M, e - (end) Mac terminates message transmission to M, br - (begin receive) M begins message delivery to Mac, er - (end receive) M terminates message delivery to Mac, b - (collision) Mac is notified that a collision has occurred on M.

EVENTS

Philips Bounded Retransmission Protocol

[D’Argenio et.al. 97]


Protocol Overview

Protocol developed by Philips.Transfer data between Audio/Video

components via infra-red communication.Data files sent in smaller chunks.Problem: Unreliable communication

medium.Sender retransmit if receiver respond too

late.Receiver abort if sender sends too late.


Overview of BRP

Sender Receiver

S R

K

L

Input: file = p1, …, pn

lossy

lossy

Output: p1, …, pn

BRP

pi

ack


How It Works

Sender input: file = p1, …, pn.

S sends (p1,FST,0), (p2,INC,1), …, (pn-1,INC,1), (pn,OK,0).

R sends: ack, …, ack.S retransmits pi if timeout.Receiver recives: p1, …, pn.Sender and Receiver receives NOK or OK.

whole file OK

more parts

will followfirst part of file


BRP Model Overview

Sender Receiver

S R

K

L

Input: file = p1, …, pn

ack

(pi,INDication,abit)

lossy

lossy

ok, nok, dkIND, ok, nok

Output: p1, …, pn

BRP


The Lossy Media

value-passing

lossy = may drop

messages

one-place

capacity

delay


Bounded Retransmission

S sends a chunk pi and waits for ack from R.If timeout the chunk is retransmitted.If too many timeout the transmission fails

(NOK is sent to Sender). If whole file successfully sent OK is sent to

Sender.Receiver is similar.


Process S


Process R


The Sender and Receiver


“If you want to know more”

Test & Verification http://www.cs.auc.dk/~ejersbo/tov/Plan.html

BRICS@Aalborg http://www.cs.auc.dk/research/FS/

UPPAAL http://www.uppaal.com

WOODDES, ATT (VHS): http://www.docs.uu.se/docs/rtmv/wooddes/ http://www-verimag.imag.fr/VHS/main.html

Strategic Directions in Computing Research Formal Methods Working Group, ACM June 1996 http://www.cs.cmu.edu/afs/cs/usr/wing/www/mit/mit.html

Test & Verification Verification Kim G. Larsen. TOV 2002Kim G. Larsen 2 Research Profile Distributed...

Documents

Transcript of Test & Verification Verification Kim G. Larsen. TOV 2002Kim G. Larsen 2 Research Profile Distributed...