Carnegie Mellon University

Formal VerificationFormal VerificationUsingUsing

Infinite-State ModelsInfinite-State Models

Formal VerificationFormal VerificationUsingUsing

Infinite-State ModelsInfinite-State Models

http://www.cs.cmu.edu/~bryant

Randal E. Bryant

Contributions by graduate students:Miroslav Velev, Sanjit Seshia, Shuvendu Lahiri

– 2 –

OutlineOutline

TaskTask Formally verify hardware and software systems Build on success in verifying finite models

Infinite-State ModelsInfinite-State Models How do they arise Need logic that is suitably expressive, yet remains

reasonably tractable.

Verification TechniquesVerification Techniques Range of methods with varying capabilities and limitations Solve problems by mapping into propositional logic

Proof engines can use powerful Boolean methods

– 3 –

Microprocessor Report, Oct. 28, 1996

Example: HP/Compaq Alpha 21264Example: HP/Compaq Alpha 21264

Pipeline StatePipeline State Multiple caches Instruction queues Dynamically-

allocated registers Memory queue Many buffers

between stages

Verification TasksVerification Tasks Does it implement

the Alpha ISA? Do specific units

satisfy desired properties?

– 4 –

Temporal Logic Model CheckingTemporal Logic Model Checking

Verify Reactive SystemsVerify Reactive Systems Construct state machine representation of reactive system

Nondeterminism expresses range of possible behaviors “Product” of component state machines

Express desired behavior as formula in temporal logic Determine whether or not property holds

Traffic LightController

Design

Traffic LightController

Design

“It is never possible to have a green light for both N-S and E-W.”

ModelChecker

True

False+ Counterexample

– 5 –

Finite System Modeling ExampleFinite System Modeling Example

Distributed, SharedMemory System

Simplifying Simplifying AbstractionsAbstractions Single word cache Single bit/word Abstract other

clusters Imprecise timing

Interface

Cluster #2Abstraction

Cluster #3Abstraction

Interface

Mem.Cache

Control.Cache

Control.

Global Bus

Cluster #1 Bus

Proc. Proc.

Arbitrary reads & writes

– 6 –

Symbolic FSM Analysis ExampleSymbolic FSM Analysis Example K. McMillan, E. Clarke (CMU) J. Schwalbe (Encore Computer)

Encore Gigamax Cache SystemEncore Gigamax Cache System Distributed memory multiprocessor Cache system to improve access time Complex hardware and synchronization protocol.

VerificationVerification Create “simplified” finite state model of system (109 states!) Verify properties about set of reachable states

Bug DetectedBug Detected Sequence of 13 bus events leading to deadlock With random simulations, would require 2 years to generate

failing case. In real system, would yield MTBF < 1 day.

– 7 –

(x1 x2) x3

Nodes represent variable testsNodes represent variable tests

Branches represent variable valuesBranches represent variable values Dashed for value 0 Solid for value 1

Canonical representationCanonical representation when reduction rules applied Makes equivalence trivial

x2

0 1

x3

x1

Boolean Manipulation with OBDDsBoolean Manipulation with OBDDs

Ordered Binary Decision Diagrams Data structure for representing Boolean functions Key to success in hardware verification

Example: Example:

– 8 –

Representing Circuit FunctionsRepresenting Circuit Functions

b3 b3

a3

Cout

b3

b2 b2

a2

b2 b2

a2

b3

a3

S3

b2

b1 b1

a1

b1 b1

a1

b2

a2

S2

b1

a0 a0

b1

a1

S1

b0

10

b0

a0

S0

FunctionsFunctions All outputs of 4-bit adder Functions of data inputs

A

B

Cout

SADD

Shared RepresentationShared Representation Graph with multiple roots 31 nodes for 4-bit adder 571 nodes for 64-bit adder Linear growth

– 9 –

Reg.File

IF/ID

InstrMem

+4

PCID/EX

ALU

EX/WB

=

=

Rd

Ra

Rb

Imm

Op

Adat

Control Control

Bdat

Simplified Processor ExampleSimplified Processor Example

Simplified RISC pipeline Register-Register and Register-Immediate operations Data hazards handled by register forwarding Each step of operation defined by function pipe

– 10 –

Reg.File

InstrMem

+4

ALU

Rd

Ra

Rb

Imm

Op

Adat

Control

Bdat

ISA Reference ModelISA Reference Model

Only programmer-visible state Much simpler control logic Assume verified against instruction set definition Each step of operation defined by function spec

PC

– 11 –

Abstracting Data from Bits to IntegersAbstracting Data from Bits to Integers

View Data as Symbolic “Terms”View Data as Symbolic “Terms” Arbitrary integers

Verification proves correctness of design for all possible word sizes

Can store in memories & registers Can select with multiplexors

ITE: If-Then-Else operation

x0

x1

x2

xn-1

x

1

0

xy

p

ITE(p, x, y)1

0

xy

T

x1

0

xy

F

y

– 12 –

Abstraction Via Uninterpreted FunctionsAbstraction Via Uninterpreted Functions

For any Block that Transforms or Evaluates Data:For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Only assumed property is functional consistency:

a = x b = y f (a, b) = f (x, y)

ALUf

– 13 –

Abstraction Via Uninterpreted FunctionsAbstraction Via Uninterpreted Functions

For any Block that Transforms or Evaluates Data:For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Also view instruction memory as function

Reg.File

IF/ID

InstrMem

+4

PCID/EX

ALU

EX/WB

=

=

Rd

Ra

Rb

Imm

Op

Adat

Control Control

F1

F 2

F3

– 14 –

Reg.File

InstrMem

+4

ALU

Rd

Ra

Rb

Imm

Op

Adat

Control

Bdat

PC

F1

F 2

F3

Abstracting Reference ModelAbstracting Reference Model

Abstract with identical functions as in pipeline model

– 15 –

EUF: Equality with Uninterp. FunctsEUF: Equality with Uninterp. Functs Decidable fragment of first order logic

Formulas (Formulas (F F )) Boolean ExpressionsBoolean ExpressionsF, F1 F2, F1 F2 Boolean connectives

T1 = T2 Equation

P (T1, …, Tk) Predicate application

Terms (Terms (T T )) Integer ExpressionsInteger ExpressionsITE(F, T1, T2) If-then-else

Fun (T1, …, Tk) Function application

Functions (Functions (FunFun)) Integer Integer Integer Integerf Uninterpreted function symbolRead, Write Memory operations

Predicates (Predicates (PP)) Integer Integer Boolean Booleanp Uninterpreted predicate symbol

– 16 –

Correctness of PipelineCorrectness of Pipeline

Abstraction Function Abstraction Function AbsAbs Relates state of pipeline to program state Result of completing partially-executed instructions

RequirementRequirement Pipeline step pipe matches k instruction executions k

spec

For our pipeline k = 1When pipeline stalls have k =0Superscalar pipelines can have k > 1

Qpipe Qpipepipe

Abs

Qspec Qspeckspec

Abs

– 17 –

Correspondence CheckingCorrespondence Checking

Burch & Dill, Computer-Aided Verification ‘94

Exploit State StructureExploit State Structure State held in memories and pipeline latches Memories match those of instruction set model Latches hold additional pipeline state

Pipeline State can be “flushed”Pipeline State can be “flushed” Control logic to support external interrupts Complete in-flight instructions Without fetching any new ones

– 18 –

Computing Abstraction FunctionComputing Abstraction Function

MethodMethod Start with arbitrary pipeline state Qpipe

Symbolically simulate processor with stall asserted Project out all but programmer-visible state

EffectEffect Processor computes its own abstraction function!

Arbitrary

Qpipepipe

Stall = 1

pipe

Stall = 1

QspecProjpipe

Stall = 1

PipelineFlushed

– 19 –

Computational Task: Single-Issue ProcessorComputational Task: Single-Issue Processor

Compare results of two symbolic simulations Starting from same initial state Number of simulation steps ~ pipeline depth

Check that resulting user-visible states identical Disjunctive acceptance condition Extra clock cycle causes either 0 or 1 new instructions to complete

Qpipe

pipe

Stall = 1

Projpipe pipe

Stall = 1 Stall = 1

pipe

Stall= 0

pipe

Stall= 1

pipe pipe

Stall= 1 Stall= 1

Proj

spec

=?=?

k=0 k=1

– 20 –

Computational Task: Dual-Issue ProcessorComputational Task: Dual-Issue Processor

Extra clock cycle causes 0, 1, or 2 new instructions to complete

spec

=?=?

k=0 k=1

Qpipe

pipe

Stall= 0

ProjFlush

Flush Proj

spec

=?

k=2

– 21 –

ALU

Ra

Rb

T = 0xa

xb

ALU

Ra

Rb

T = 1xa

xb

f

Term-Level Symbolic SimulationTerm-Level Symbolic Simulation

ALU

Ra

Rb

T = 2xa

xb

f f

ALU

Ra

Rb

T = 3xa

xb

f f f

Simulator OperationSimulator Operation Register states are term-level expressions

Denoted by pointers to nodes in Directed Acyclic Graph (DAG)

Simulate each cycle of circuit by adding new nodes to DAGBased on circuit operations

Construct DAG denoting correctness condition

– 22 –

Decision ProblemDecision ProblemLogic of Equality with Uninterpreted Functions (EUF)Logic of Equality with Uninterpreted Functions (EUF)

Truth ValuesDashed LinesModel ControlLogical connectivesEquations

Integer ValuesSolid linesModel DataUninterpreted functions If-Then-Else operation

TaskTask Determine whether formula is universally valid

True for all interpretations of variables and function symbols

=

f

T

F

T

F

fT

F

=

e1

e0x0

d0

– 23 –

Finite Model Property for EUFFinite Model Property for EUF

ObservationObservation Any formula has limited number of distinct expressions Only property that matters is whether or not different terms

are equal

=

f

T

F

T

F

fT

F

=

e1

e0x0

d0

x0 d0 f (x0) f (d0)

– 24 –

Boolean Encoding of Integer ValuesBoolean Encoding of Integer Values

For Each ExpressionFor Each Expression Either equal to or distinct from each preceding expression

Boolean EncodingBoolean Encoding Use Boolean values to encode integers over small range EUF formula can be translated into propositional logic

Tautology iff original formula valid

ExpressionExpression Possible Possible ValuesValues

Bit Bit EncodingEncoding

x0 {0}{0} 00 00

d0 {0,1}{0,1} 00 bb1010

f (x0) {0,1,2}{0,1,2} bb2121 bb2020

f (d0) {0,1,2,3}{0,1,2,3} bb3131 bb3030

– 25 –

Benchmark CircuitsBenchmark Circuits

Single Issue Pipeline: Single Issue Pipeline: 1xDLX1xDLX Analogous to DLX model in Hennessy & Patterson Verified in ‘94 by Burch & Dill

Dual Issue Pipeline : Dual Issue Pipeline : 2xDLX-CC2xDLX-CC Superscalar operation with two complete pipelines

Full-Featured Pipeline: Full-Featured Pipeline: 2xDLX-*2xDLX-* Multi-cycle function units, exception handling & branch

prediction

– 26 –

EvaluationEvaluation

Using BDD Evaluation to Prove TautologyUsing BDD Evaluation to Prove TautologyCircuit BDD Vars. BDD Nodes CPU Secs.1xDLX 63 2,127 0.22xDLX-CC 173 51,826 202xDLX-* 418 986,740 2,635

Using SAT Checkers to Prove TautologyUsing SAT Checkers to Prove Tautology Chaff (Malik, Princeton) Major advances in last few yearsCircuit CNF Vars. Clauses CPU Secs.2xDLX-* 4,583 41,704 22

– 27 –

An Out-of-order Processor (OOO)An Out-of-order Processor (OOO)

Data Dependencies Resolved by Register RenamingData Dependencies Resolved by Register Renaming Mapping from register ID to instruction in reorder buffer that will

generate register value

Inorder Retirement Managed by Retirement BufferInorder Retirement Managed by Retirement Buffer FIFO buffer keeping pending instructions in program order

Reorder BufferFields

PC

Programmemory

Reorder Buffer

validvaluesrc1validsrc1valsrc1tagsrc2validsrc2valsrc2tagdestop

result bus

DECODE Register

Rename Unit

valid tag val

ALU

head tail

incrdispatch

retire

execute

result

1st

Operand

2nd

Operand

– 28 –

ALU

execute

Directly AddressableDirectly Addressable Select particular entry for

execution Retrieve result value from

executed instruction

Access Modes for Reorder BufferAccess Modes for Reorder Buffer

FIFOFIFO Insert when dispatch Remove when retire

Content AddressableContent Addressable Broadcast result to all

entries with matching source tag

head tail

Retire Dispatch

result bus

GlobalGlobal Flush all queue entries when

instruction at head causes exception

– 29 –

Required LogicRequired Logic

Increased Expressive PowerIncreased Expressive Power Model queue pointers

Increment & decrement operationsRelative ordering

Ability to construct complex memory structuresNot just set of fixed memory types

Don’t Go Too FarDon’t Go Too Far Want practical decision procedures Efficient reduction to propositional logic

– 30 –

EUF CLUEUF CLUTerms (Terms (T T ))

ITE(F, T1, T2) If-then-else

Fun (T1, …, Tk) Function application

Formulas (Formulas (F F ))F, F1 F2, F1 F2 Boolean connectives

T1 = T2 Equation

P(T1, …, Tk) Predicate application

succ (T) Increment

pred (T) Decrement

T1 < T2 Inequality

– 31 –

EUF CLU (Cont.)EUF CLU (Cont.)

Functions (Functions (FunFun))f Uninterpreted function symbol

Read, Write Memory operations

Predicates (Predicates (PP))p Uninterpreted predicate

symbol

x1, …, xk . T Function lambda expression

x1, …, xk . F Predicate lambda expression

• Arguments can only be terms• Lambdas are just mutable arrays

– 32 –

Modeling Memories with ’sModeling Memories with ’s

Memory M Modeled as FunctionMemory M Modeled as Function

M(a): Value at location a

InitiallyInitially

Arbitrary state Modeled by uninterpreted

function m0

Writing Transforms MemoryWriting Transforms Memory M = Write(M, wa, wd)

a . ITE(a = wa, wd, M(a)) Future reads of address wa

will get wd

Ma

M

a m0

M

Ma 1

0

wd

=wa

– 33 –

Modeling Unbounded FIFO BufferModeling Unbounded FIFO Buffer

Queue is Subrange of Infinite SequenceQueue is Subrange of Infinite Sequence Q.head = h

Index of oldest element

Q.tail = t Index of insertion location

Q.val = qFunction mapping indices to valuesq(i) valid only when h i < t

Initial State: Arbitrary QueueInitial State: Arbitrary Queue Q.head = h0, Q.tail = t0

Impose constraint that h0 t0

Q.val = q0

Uninterpreted function

q(h–2)

q(h–1)

q(h)

q(h+1)

•••

q(t–2)

q(t–1)

q(t)

q(t+1)

•••

•••

tailtail

headhead

AlreadyPopped

Not YetInserted

incr

ea

sin

g in

dic

es

– 34 –

Modeling FIFO Buffer (cont.)Modeling FIFO Buffer (cont.)

tt

q(h–2)

q(h–1)

q(h)

q(h+1)

•••

q(t–2)

q(t–1)

q(t)

q(t+1)

•••

•••

hh

next[h] := ITE(operation = POP, succ(h), h)

next[q] := (i).ITE((operation = PUSH & i=t), x, q(i))

next[t] :=ITE(operation = PUSH, succ(t), t)

q(h–2)

q(h–1)

q(h)

q(h+1)

•••

q(t–2)

q(t–1)

x

q(t+1)

•••

•••

next[t]next[t]

next[hnext[h]]

op = PUSHInput = x

– 35 –

Decision ProcedureDecision Procedure

OperationOperation Series of

transformations leading to propositional formula

Propositional formula checked with BDD or SAT tools

Bryant, Lahiri, Seshia [CAV02]

LambdaExpansion

Function&

PredicateElimination

Convert to BooleanFormula

BooleanSatisfiability

CLUFormula

-freeFormula

Function-freeFormula

BooleanFormula

– 36 –

Finite Model Property for CLUFinite Model Property for CLU

ObservationObservation Need to encode all possible relative orderings of

expressions Each symbolic value has maximum range of increments &

decrements Can use Boolean encodings of small integer ranges

x y succ(x) > pred(y)

x x+1

y –1 y y –1 y

x x+1

y –1 y

x x+1

y –1 y

x x+1 x x+1

y –1 y

x = 0, y = 3 x = 2, y = 1

– 37 –

Verification Techniques in UCLIDVerification Techniques in UCLID

Bounded Property CheckingBounded Property Checking Start in reset state Symbolically simulate for fixed number of steps Verify a safety property for all states reachable within the

fixed number of steps from the start state

Correspondence Checking Correspondence Checking Run 2 different simulations starting in most general state Prove that final states equivalent e.g. Burch-Dill Technique

Invariant CheckingInvariant Checking Start in general state s Prove Inv(s) Inv(next[s]) Limited support for automatic quantifier instantiation

– 38 –

Verification of OOO : Automation vs. GuaranteeVerification of OOO : Automation vs. Guarantee

Presence of decision procedurePresence of decision procedure Efficiency : Allows improved bounded property checking

and Burch-Dill method Automation : Reduces manual guidance in proving

invariants Automatic Instantiation of quantifiers

Method Resources Verification (# of steps)

Auxiliary variables

Invariants

Bounded Property Checking

Unbounded Bounded None None

Burch-Dill Technique

Fixed Unbounded None Very few

Inductive Invariant Checking

Unbounded Unbounded Significant Significant, including those for auxiliary variables

– 39 –

Technique 1 : Bounded Property CheckingTechnique 1 : Bounded Property CheckingDebugging OOO using Bounded Property CheckingDebugging OOO using Bounded Property Checking

All the errors were discovered during this phase Counterexample trace of great help

Debugging Motorola ELF™ Debugging Motorola ELF™ Superscalar out-of-order processor Reorder Buffer, memory unit, load-store queues etc. Applied during early design exploration phase

– 40 –

Bounded Property Checking ResultsBounded Property Checking Results

SVC (Stanford) : Another decision procedure to solve CLU formulas SVC (Stanford) : Another decision procedure to solve CLU formulas Can decide more expressive class

CVC (Successor of SVC) runs out of memory on larger casesCVC (Successor of SVC) runs out of memory on larger cases

ModelModel stepssteps termsterms Term Term formula formula sizesize

Prop Prop Formula Formula SizeSize

UCLID UCLID time (s)time (s)

SVC time SVC time (s)(s)

OOO unitOOO unit 10 59 2566 15290 10.8 233.18

14 87 7480 62504 76.55 > 5 hrs

20 129 19921 263413 1679.12 > 1 day

Elf™Elf™ 6 33 218 942 1.2 10.9

8 70 1085 4481 8.4 1851.6

10 104 2467 16453 30.6 > 1 day

12 149 4553 54288 111.0 > 1 day

– 41 –

Burch-Dill Technique for OOOBurch-Dill Technique for OOO

Exponential blowup with the number of ROB entriesExponential blowup with the number of ROB entries Limited to r = 8 entries currently r = 8 finished after case-splitting in 2.5hrs

# Of ROB# Of ROB

EntriesEntries

# of # of termsterms

Term Term formula formula

sizesize

Prop Formula Prop Formula SizeSize

UCLID UCLID time (s)time (s)

2 63 398 5325 6.83

3 83 618 10248 30.23

4 103 886 18175 157.41

6 143 1534 41208 3051.79

8 183 2342 82915 >31hrs

– 42 –

Technique 3 : Invariant CheckingTechnique 3 : Invariant Checking

Deriving the inductive invariantsDeriving the inductive invariants Require additional (auxiliary) variables to express invariants Auxiliary variables do not affect system operation

Proving that the invariants are inductiveProving that the invariants are inductive Automate proof of invariants in UCLID Eliminates need for large (often fragile) proof script

– 43 –

Restricted Invariants and ProofsRestricted Invariants and Proofs

Restricted classes of invariantsRestricted classes of invariants x1x2…xk (x1…xk)

(x1…xk) is a CLU formula without quantifiers

x1…xk are integer variables free in (x1…xk)

Proving these invariants requires quantifiersProving these invariants requires quantifiersx1x2…xk (x1…xk) y1y2…ym (y1…ym)

x1 x2…xk y1y2…ym [(x1…xk) (y1…ym)]

Automatic instantiation of Automatic instantiation of x1…xk with concrete termswith concrete terms Sound but incomplete method

Reduce the quantified formula to a CLU formulaReduce the quantified formula to a CLU formula Can use the decision procedure for CLU

– 44 –

Proving InvariantsProving Invariants

Proved automaticallyProved automatically Quantifier instantiation was sufficient in these cases Relieves the user of writing proof scripts to discharge the

proofs Time spent = 54s on 1.4GHz m/c Total effort = 2 person days

– 45 –

Extending the DesignExtending the Design

baseExecutes ALU instructions only

excHandles arithmetic exceptionsMust flush reorder buffer

exc/brHandles branchesPredicts branch & speculatively executes along path

exc/br/mem-simpAdds load & store instructionsStore commits as instruction retires

exc/br/memStores held in bufferCan commit laterLoads must scan buffer for matching addresses

– 46 –

Comparative Verification EffortComparative Verification Effort

base exc exc / br exc / br / mem-simp

exc / br / mem

Total Invariants

13 34 39 67 71

Manually instantiate

0 0 0 4 8

UCLID time

54 s 236 s 403 s 1594 s 2200 s

Person time

2 days 5 days 2 days 15 days 10 days

– 47 –

Beyond Processor VerificationBeyond Processor Verification

Systems of Identical ProcessesSystems of Identical Processes E.g., synchronization protocols Arbitrary number of processes, each having same operation

SoftwareSoftware Create finite model by predicate abstraction

– 48 –

Systems of Identical ProcessesSystems of Identical ProcessesEach Process has Each Process has kk State Variables State Variables

Each state variable represented as array Indexed by process Id

• • • ••• sv1• • • ••• sv2

• • • ••• svk

State of Process i

– 49 –

Modeling System of Identical ProcessesModeling System of Identical ProcessesOn Each Step:On Each Step:

Select arbitrary process index pAs if chosen by nondeterministic scheduler

Update state for selected process

next[state] := lambda(i)

case

i = p & state(i) = IDLE: TRYING

i = p & state(i) = TRYING & inuse : TRYING

i = p & state(i) = TRYING & !inuse: CRITICAL

default: state(i)

esac

IDLE

TRYING

CRITICAL

• • • ••• state 0/1

inuse p

– 50 –

Model Checking SoftwareModel Checking Software

Program is Hard to Model as Finite-State MachineProgram is Hard to Model as Finite-State Machine Large number of large data words means lots of bits

Although “finite”, bound is very large

Recursion requires stackConceptually unbounded

Creating Finite State AbstractionCreating Finite State Abstraction Microsoft SLAM verifier

Focus on device drivers

Start with very abstract model of programEvery conditional can arbitrarily be taken/not-taken

Check propertiesE.g., always close files

Refine when find counterexampleMore careful analysis of conditionals

– 51 –

Code Verification ExampleCode Verification Example

Adapted by Tom Ball from PCI device driver code Initial verification run based on simple model of control flow

do { lock(v); old = new; if (test()) { unlock(v); new++; } } while (new != old);unlock(v);

Properties to CheckProperties to Check Cannot unlock v unless

locked Cannot lock v unless

unlocked Must exit code with v

unlocked

– 52 –

Model as Boolean ProgramModel as Boolean Program

All conditionals abstracted as Boolean variables Allows arbitrary branching

Finite-state approximation of program

do { lock(); if (a) { unlock(); } } while (b);unlock();

Apparent bug: May call unlock twice

Apparent bug: May call lock twice

– 53 –

Refining AbstractionRefining Abstraction

Add more detail to model to prove that errors do not occur Use lightweight theorem prover to check

do { lock(); old = new; if (test()) { unlock(); new++; } } while (new != old);unlock();

old = new

do { lock(); if (a) { unlock(); } } while (b);unlock();

Double locking

!a !b

!a

b

– 54 –

Refining Abstraction (cont.)Refining Abstraction (cont.)

Continue using counterexamples to generate more constraints on allowed state transitions

do { lock(); old = new; if (test()) { unlock(); new++; } } while (new != old);unlock();

Double unlocking

do { lock(); old = new; if (a) { unlock(); new++; } } while (b);unlock();

old new

old = new

a b

a

!b

– 55 –

Software Verification StatusSoftware Verification Status

Shows PromiseShows Promise Reason about real-life code Fully automatic

No user-supplied assertions or induction hypotheses

Still in Early StagesStill in Early Stages Can only deal with limited class of programs

Memory referencing & aliasing possibilities difficult to decipher

Look for particular classes of errorsProperty checking rather than comprehensive verification