Verification & Validation

By: Amir Masoud Gharehbaghi

Email: amgh@mehr.sharif.edu

2V&V Techniques

Outline

Overview Selective History Theorem Proving Model Checking Hardware Verification Assertion-Based Verification Conclusions

3V&V Techniques

What is Verification (Validation)

Functional Verification:

Task of establishing that a given design accurately implements the intended behavior

4V&V Techniques

Verification&Validation Techniques

Simulation-basedApply inputs to design, simulate (or run), and

check the results Formal

Mathematically proof the correctness of system against the properties

Semi-formalCombine simulation and formal verification

5V&V Techniques

V&V Techniques Comparison

Simulation-basedEasy to useFastLow coverage

FormalPerfect coverageNot easy to useNot applicable for large designs

6V&V Techniques

Using Formal Methods

Writing formal specifications Proving properties about the specification

Theorem proving

Deriving implementation from a given specification Refinement

Verifying properties for a given implementation Property checking

7V&V Techniques

Selective History

Early 1960’s Suggestions: McCarthy and Dijkstra

Late 1960’s and Early 1970’s Proof systems: Floyd-Hoar, Boyer-Moore,…

Late 1970’s Temporal Logic for reactive systems (Pnueli, …)

Early 1980’s Model checking (Clarke, Emerson, …)

Late 1980’s Symbolic model checking using BDDs

8V&V Techniques

Selective History (cont.)

1990’s -> mostly hardware Non-BDD based model checking Satisfiability Equivalence checking Symbolic simulation & symbolic trajectory evaluation

2000’s Assertion-based verification Software model checking Probabilistic verification Automated theorem proving Hybrid systems verification

9V&V Techniques

Theorem Proving

Formally specify the system in a logic system

Formally specify the properties of system Prove the correctness of properties of

system in a proof system

10V&V Techniques

Theorem Provers & Logic Systems

First-Order LogicACL2Nqthm

High-Order LogicHOLPVS

11V&V Techniques

Temporal Logic

First-Order Logic + Temporal Operations

Linear Temporal Logic (LTL) Computational Tree Logic (CTL)

12V&V Techniques

LTL Temporal Operations

X: next F: finally G: globally U: until

13V&V Techniques

LTL Example

p X p alert F halt G (alert F halt ) G (alert (alarm U halt ) )

14V&V Techniques

CTL Path Operations

A: always E: there exists

Combine with temporal operations of LTL:AX, AF, AG, AUEX, EF, EG, EU

15V&V Techniques

CTL Example

AG p AF halt E ( alaram U halt ) AX alarm EF close

16V&V Techniques

Properties

Safety Something never occurs.

Liveness Something will ultimately occur.

Reachability Some particular situation can be reached.

Fairness Something will (not) occur infinitely often.

Properties are checked under certain conditions

17V&V Techniques

Model Checking

Model is a state machine (or automata) Property is defined in a temporal logic

CTL model checking O(|Q| * |p|)

LTL model checking O (|Q| * 2^|p| )

|Q|: number of states |p|: number of sub-formulas in property

18V&V Techniques

State Explosion Problem

Both in LTL and CTL:An automata is generated (explicitly)

Number of states grow exponentially

19V&V Techniques

Symbolic Model Checking

Symbolically (implicitly) represent states (and transition between states)

Use Binary Decision Diagram (BDD) to represent state variables

Uses CTL properties

20V&V Techniques

Reduced Order BDD (ROBDD)

A directed acyclic graph (DAG) with two leaf nodes (1,0)

Represent Boolean functionsCompactCanonicalEfficient operations (linear or quadratic)Simple to use

21V&V Techniques

Satisfiability (SAT) Checking

Satisfiability Checking: Check existence of a combination of values

for a Boolean function that function is 1 Check that ~f is unsatisfiable

22V&V Techniques

Bounded Model Checking

Search for counter example by unfolding system in time until a bound is reached.

Use SAT checkers

What about unbounded model checking?

23V&V Techniques

Symbolic Simulation

Simulate with symbolic (not explicit) values. Inputs: expressionsOutputs: expressions

Originally based on BDD.

24V&V Techniques

Symbolic Trajectory Evaluation

Check properties of A=>C form. A: input variables’ values over time C: expected output variables’ values over time

Symbolically simulate with given input values (A).

Check that expected results (C) to be compatible (subset of) simulated output results.

25V&V Techniques

Equivalence Checking

Check equivalent behavior between two designsSame level of abstraction Different levels of abstraction

Combinational Sequential

26V&V Techniques

Equivalence Checking Methods

Combinational Compare the canonical representation of two circuits. (may be

not feasible) Use SAT checker …

Sequential Find equivalent FFs and Compare combinational circuits

between them. Construct the multiplicative state machine and check the

equivalency of outputs in all states. Bounded model checking

27V&V Techniques

Assertion-Based Verification

Assertion: property

Do property checking during simulationEmbed in designCheck in run-time

28V&V Techniques

Assertion Languages

OVL: Open Verification library PSL: Property Specification Language

Formerly “Sugar” SystemVerilog …

29V&V Techniques

Coverage

Percentage of design covered during simulation Code

Statement Path Condition …

Signal …

30V&V Techniques

Conclusions

Verification is a serious bottleneck for current designs Up to 80 percent of design time

Formal methods cannot be applied to real designs

Simulation cannot guarantee correctness of designs

Embedded system verification containing Hw/Sw requires new techniques