Accelerating the Industrial Deployment of Model-Checking for SW Verification: Lessons from 10 years...

Accelerating the Industrial Deployment of Model-Checking for SW Verification:

Lessons from 10 years of Model Checking Deployment for HW Verification in Intel

Limor Fix

Intel Research

Agenda

Acknowledgements

Brief History

Hardware formal verification at Intel – the first generation

Next generation

Discussion – industrial deployment of software model checking

1995

2004

Acknowledgements

The success of formal verification in Intel was the result of a

great collaboration between industry and academia

My team has worked with Randy Bryant, P.P. Chakrabarti, Ed Clarke, P. Dasgupta, David Dill, Enrico Giuchilia, Orna Grumberg, Scott Hazelhurst, Sharad Malik, Zohar Manna, Amir Pnueli, Assaf Schuster, Fabio Somenzi, Armando Tacchella, Moshe Vardi, and Lenore Zuck

Brief History First generation FPV product (Athena): 1995-1999

Tool “Prover” (FSL+SMV)Used by 2 lead CPU (Merced, WMT) during 95-99WMT post-mortem: very successful, important High-Quality-Bugs (HQB) were found, ¼ of

validation team was FV Merced post-mortem: important HQB

Second generation FPV product (Nike): 2000-2002Tool “FPV system”

ForSpec (property language), FedEx (ForSpec checkers), Forecast (BDD based model checker), Thunder (SAT based Bounded/Full model-checker), Forte(STE, FL)

Users: Lead CPU designs, Chipsets, communication chips Third generation FPV product: 2003-2005

Tool “Foresight” ForSpec (standalone and embedded), Libraries of properties (FSTL), Thunder,

FedEX+, ForesightUsers: Lead CPU designs, Chipsets, communication chips

Fourth generation: 2005-…

Prover: The First Generation

1995 – 1999: the deployment of the first formal property verification in Intel

Prover: The first generation of Intel formal property verification system

Prover included:

Enhanced version of SMV FSL: a specification language that was an hardware linear

temporal language inspired by LTL The compiler for FSL translated the linear logic into automata

using tableau-like algorithms FSL was used both:

to specify formal properties to be verified by the model checker and

to specify checkers to be checked dynamically during simulation of the hardware designs.

Prover: The first generation

Two lead CPU design teams used Prover for 4 years (in Santa Clara and Oregon)

Both teams reported successful usage, high quality bugs have been found. Either, these bugs were likely to be found by other validation tools much later in

the design cycle Or, these bugs might have escaped all the validation tools and reach the silicon

Very important leanings were generated by the two design teams

about: How Where When By whom

Also, the remaining technology challenges were identified

How

Automated abstraction (vs. Manual Abstraction) Property based pruning Input elimination and data abstraction Manual abstraction: did not fly

Modular verification using assume-guarantee paradigm Partitioning the design following its syntactic structure Partitioning the design following the its functionality Partitioning continued till a certain count of states was reached

Properties were developed to capture the intended behavior of

the inputs and the outputs of each module

Desired properties sometimes ended up as a large set of smaller assumptions and properties

Where

Only selected areas of the CPU were formally verified

These were areas of high risk in which new complex functionality was added:

Memory Cluster

Bus Cluster

Execution Cluster

Front End Cluster

When

Early in the design cycle, even before simulation model is ready

Pro: Bugs were revealed early and did not even reach the early simulation models

Con: RTL was unstable at that time and the RTL changes required recoding of the properties and assumptions again and again

The teams ended up using the tools relatively late and have indicated that support for early verification would be very beneficial

By Whom

Both the validation and the RTL-design groups used Prover

Number of users: More validators than designers

Effectiveness: Designers used Prover more effectively

Challenges Identified

Limited capacity of the model checker

Assumption handling Hard to know which assumptions were needed Hard to identify circular reasoning Hard to make sure all assumptions were verified

Development of high quality specification was very difficult

Hard to develop high level properties that do not mimic the details of the implementation,

Hard to train the designers to use a linear temporal language Impossible to know if enough properties were developed to express the

entire desired behavior of the design, Hard to maintain the properties since the design was changing

Challenges Identified

Need to replace current validation activity or at least integrate formal verification into current design or validation activity

Simulation only - RTLer flow

RTL development / update

MASold/wwXX

RTL

RTL check-in

mini regressionRTL CTEsimulation

RTLwwXX+1

checked in

Cluster validation flowFull Chip

validation flow

Debug using

bug?

yes no

build test and run

RTL simulation

bugs

Simulation Only - Cluster Validator’s Flow

Ramp on Design

MAS/RTL

Develop/update test-plan

Develop CTE environment

Develop CTE checkers

Develop Coverage

Build & Run CTE

Analyze Checkerresults

Analyze coverageresults

RTLer

RTLer flow

bugs

CoverageDB

Test DB

Simulation Only – Full Chip Validator’s Flow

Develop/update test-plan

Develop/update architectural

checker

Develop testsDevelop/updateother checkers

Build & Run FC

Analyze Checkerresults

Analyze coverageresults

RTLer flow

bugs

CoverageDB

TestDB

Next Generations In the following years:

New specification language (ForSpec)

Libraries of properties

Compilers that automatically generate properties and assumptions

Multiple engines: SAT-based model checker Symbolic simulation engine Parallel model checker

Formal specification coverage tools Measure the completeness of the set of properties that have been developed

Database of properties and assumptions to help detect circular reasoning and track the status of all properties

Verification manager

Verification tools for microcode

Next Generations – Property Language ForSpec

ForSpec has two versions: A standalone version in which the properties are developed in a separate

file

An embedded version in which properties are developed as embedded assertions inside the RTL model, that is, as part of the Verilog code

PSL and SVA IEEE standards Intel has donated ForSpec to AccelleraThe resulting PSL IEEE standard, has adopted major parts of

ForSpec standalone version. The resulting SVA IEEE standard for SystemVerilog assertion

has also adopted major parts of ForSpec embedded version

RTLer flow with Foresight

RTL development / update

MASold/wwXX

RTL

RTL check-in

mini regressionRTL CTEsimulation

RTLwwXX+1

checked in

Cluster validation flow FC validation flow

Debug

bug?

yes no

Assertionsinsertion

checkingassertions

checkingassertions

build test and run

RTL simulation

(semi)-Formalverification of

assertions

coverageDB

Test /properties

DB

bugs

Summary of the deployment of formal property verification in Intel

Took five years to move from experimental deployment to wide deployment

Close interaction with early adopters – a must

Understanding the current flows and challenges – a must

Industrial standards – very important

Major success

Centrino validator: “FV saved us a lot of Debug effort. FV cleaned the XXX bugs inside the modules and left us to deal only with the Full Chip context bugs. Given the circumstances ……… and other difficulties ………, without FV cleaning the bugs inside the modules before we even started, I think we couldn’t make it to Tap-Out on time”

Industrial deployment of model checking for SW verification

Need to think about: How Where When By Whom

How

Embedded assertions Have been proven very successful in hardware verification Need to be dense enough Generate some of them automatically Develop a library of parameterized assertions Embedded assertions need to be utilized for several propose:

compiler optimization debugging using gdb-like debuggers

Database of assertions need to be generated automatically from the program code for managing the status of the assertions

False negativeUse assertion as run time checkersFocus first on previously passing assertionsTowards “tap out” clean all failing assertions

Where

Parallel programs Most state-of-the-art computing devices have multiple processing units

and the move to chip multiprosessors is happening very fast. New programming paradigms are being developed to combat the

difficulties of parallel programming. The new developed languages should include embedded assertions as

integral part of the language Security

The problem of viruses, spyware, and worms is growing fast and has very high costs.

Extra efforts to reduce vulnerability of software are likely to be invested to prevent these costs.

When

Assertions development should become an integral part of writing (and Compiling) software

By Whom

Most assertions need to be developed by the programmers themselves and should be always turned on

The validation teams may add more assertions later and most importantly they need to work with the assertion database to complete the verification of all assertions.

Thank You

Accelerating the Industrial Deployment of Model-Checking for SW Verification: Lessons from 10 years...

Documents

Transcript of Accelerating the Industrial Deployment of Model-Checking for SW Verification: Lessons from 10 years...