Coverage

SystemVerilog Verification Lecture 11

Coverage

1/28/2013 Krispan Inc. Confidential 1

Coverage types

• Coverage is a generic term used to measure the progress of verifying a design. Two types of Coverage Code and Functional

– Code coverage:

• Here you are measuring how many lines of code have been executed(line coverage).

• Which paths through the code and expressions have been executed(path coverage)

• Which states and transitions have been executed(FSM)

• Single bit variable transitions from 0 or 1 (toggle coverage)

• Most Simulators include a code coverage tool.

• Code Coverage checks how accurately your tests exercised the implementation of the design specifiacation and not the verification plan.

• 100% code coverage does not mean your design is completely verified


Code Coverage

• The code coverage tool will not catch mistakes you make – Suppose your design implementation is missing a feature from the

design specification. Code Coverage will not catch it.

– Suppose you forget to implement the reset logic in your flop. Code coverage will not catch it. It will report that every line in the code is exercised.

module dff ( output logic q,

input logic clk, d, reset_low);

always @(posedge clk or negedge reset_low) begin

q <= d;

end

endmodule

• The reset logic was not implemented. Code Coverage tool will not catch it.


TestBench with Functional Coverage


test

Generator

Driver

DUT

Assertions

Scoreboard checker

monitor

Test

Environment

Functional C

overa

ge

Functional Coverage

• Functional Coverage tells you how much of the design is exercised.

• If the design is exercised completely for all possible scenarios then the coverage is 100%.

• Create a test plan with the list of tests that are needed from the design specification.

• Look at the coverage results, then decide on what actions one needs take in order to achieve 100% coverage

• First action is to run the existing tests with more seeds to see if you can achieve 100% coverage.

• Next step is to write more constraints and see if that fulfils the cause.

• Write Directed tests only if absolutely necessary


Coverage Convergence

• For e.g. if we are looking at coverage metrics for processors

– Did we cover all instructions

– Did we cover all addressing modes

– Did we cover all valid combinations of instructions and addressing modes.

• Functional coverage identifies tested and untested areas of your design and makes sure you test for all

– Corner cases

– All states in a state machine and transitions are verified

• Use a combination of seeds, constraints and directed tests if needed to increase your coverage to 100%.


Functional Coverage


Design Specification Verification Plan

design tests Coverage

database

pass

Coverage

Analysis Debug

Functional Coverage

• You can see from the previous diagram you test for pass/fail

of a test. – Pass gives you the coverage analysis

– Fail has no significance

– Functional Coverage data is discarded if the test failed due to design bug.

– The Coverage data gives you the verification metrics and gives you a measure of how many items in the verification plan is complete which is based on the design specifications

– If the design does not match the specifications then the coverage values have no meaning.

– Analyze coverage with multiple runs with different seeds, more constraints, and directed tests if needed to achieve the coverage goal.


covergroup

• As we have seen before functional coverage starts with writing of the verification plan from design specification

• Then in SystemVerilog we write an executable version using covergroup and cover points

• covergroup construct is a keyword that encapsulates the specification of a coverage model. It may include a clocking event that samples values of variables and expressions which are called cover points.

• Cover groups are defined using the keyword covergroup and endgroup.

• A covergroup instance can be created using the new operator.


covergroup

• Coverage model definiton – Encapsulated in the covergroup construct defined by the user

– Clocking events for sampling variables or expressions which are cover points . This is done using the keyword coverpoint

– Cross coverage between coverage points using keyword cross

– Optional coverage options

• SystemVerilog functional coverage enables

– Variable and expression coverage as well as cross coverage

– Automatic and/or user-defined coverage bins

– Filtering conditions (illegal values, values to ignore, etc.)

– Automatic coverage sample triggering by events and sequences

• – Dynamic query of coverage information via built-in methods


covergroup

• Syntax is covergroup buslogic;

……..

……..

endgroup

buslogic bl;

bl = new;

• An instance of buslogic covergroup bl is created by assigning bl to new


covergroup example

program test1(busifc.TB ifc)

class BusTransaction;

//properties variables (data portion)

rand bit [31:0] data; //random variables

rand bit [3:0] port; //16 possible ports

endclass

covergroup portdata

coverpoint bt.port; // Measure coverage

endgroup

initial begin

BusTransaction bt;

portdata pd;

pd = new(); // Instantiate group

bt = new();

repeat (32) begin // Run a few cycles

assert(bt.randomize); // Create a transaction

ifc.cb.port <= bt.port; // and transmit

ifc.cb.data <= bt.data; // onto interface

pd.sample(); // Gather coverage

@ifc.cb; // Wait a cycle

end end endprogram


covergroup example

• In this example a random transaction is created and driven to the interface.

• Testbench samples the value of the port field using the portdata covergroup.

• 16 possible values, 32random transactions. Did your testbench have enough coverage

• vcs will generate a coverage report which gives you a summary of your coverage.

• If coverage does not satisfy your requirements then you can rerun with a new seed value, add more constraints etc.


Sample of vcs® output

• Coverpoint Coverage report

• CoverageGroup: portdata

• Coverpoint: bt.port

• Summary

• Coverage: 50

• Goal: 100

• Number of Expected auto-bins: 16

• Number of User Defined Bins: 0

• Number of Automatically Generated Bins: 7

• Number of User Defined Transitions: 0

• Automatically Generated Bins

• Bin # hits at least

• ================================

• auto[1] 7 1

• auto[2] 7 1

• auto[3] 1 1

• auto[4] 5 1

• auto[5] 4 1

• auto[6] 2 1

• auto[7] 6 1

• ================================


Sample of vcs® output

• This is just how the vcs output would look like. It is not a real output.

• You can see testbench generated the values for ports 1,2,3,4,5,6,7 but nothing was generated for 0,8,9,10,11,12,13,14,15.

• The at least option specifies how many hits are needed before a bin is covered.

• The easiest way to improve coverage is to try to generate more random transactions (more simulation cycles) or to try new seed values.


Coverage bins

• Bins can be created implicitly or explicitly

• When you specifiy coverpoints if you do not specify any bins then implicit bins are created.

• In the example vcs file we were trying the coverage on each port and since there are 16 ports it will automatically create 16 bins

• The number of bins to be created can be controlled by using the auto_bin_max parameter.

covergroup portdata

coverpoint bt.port;

{ option.auto_bin_max = 8; }

endgroup

• In this case total bins created will be 8 half of 16 and 1 bin for 2 ports


Coverage Bins

• Explicit bin creation is the preferred way. Not all cover points are of interest to the user so he can use explicit bins if he knows exactly what he wants to cover. You can also name the bins

covergroup portdata

coverpoint bt.port {

bins portzero = {0}; // 1 bin for port0

bins portone = {[1:5], 7}; // 1 bin for ports 1:5, and 7

bins porttwo = {6, [8:15]}; // 1 bin for ports 6, 8:15

}

endgroup // portdata

• In this case we created 3 bins as shown above.


Conditional Coverage

• You can add the iff keyword to add a condition to the coverpoint covergroup portdata

coverpoint bt.port iff(!bus.if.reset)

endgroup

• This way you can turn off cover for ports during reset assuming reset is active high.

• you can also use start and stop functions to turn on and off cover.

• Here is an example of that


Conditional Coverage

initial begin

BusTransaction bt;

portdata pd;

pd = new(); // Instantiate group

bt = new();

#1 pd.stop(); //stop coverage

bus.if.reset = 1; //start reset

#100 bus.ifi.reset = 0; //end reset

pd.start(); //start coverage

repeat (32) begin // Run a few cycles

assert(bt.randomize); // Create a transaction

ifc.cb.port <= bt.port; // and transmit

ifc.cb.data <= bt.data; // onto interface

pd.sample(); // Gather coverage

@ifc.cb; // Wait a cycle

end

end


Covergroup range

• You can use the $ to specify the upper limit if it is used to specify the right side of the range to specify the upper limit

• You can use the $ to specify the lower limit if it is used on the leftside of the range to specify the lower limit

covergroup portdata


bins portzero = {[$:5]}; // 1 bin for port 0:5

bins portone = {[6,7]}; // 1 bin for ports 6 and 7

bins porttwo = {[8:$]}; // 1 bin for ports 8:15

}

endgroup // portdata


Creating bins for enumerated types

• For enumerated data types SystemVerilog creates a bin for each value of the enumerated type

typedef enum { idle, decode, data_transfer} fsmstate;

fsmstate cstate, nstate;

covergroup cg_fsm;

coverpoint cstate;

endgroup

• In this case it will create a bin each for idle, decode and data_transfer.

• If you want to group multiple values into a single bin then you have to define your bins just as we did before


Transition Coverage

• Individual State transitions can be specified for a cover point.

• For eg you can check if port went from 0 to 1 2 or 3

covergroup portdata


bins p1 = (0 => 1), (0 => 2), (0 => 3); } // Measure coverage

endgroup

• Wildcard states and transitions:

– Wildcard keyword is used to specify state transitions. X,Z or ? are treated as a wildcard for 0 or 1. Here is a way to create a coverpoint with bins for even and odd values

bit [2:0] port;

covergroup cp;

coverpoint port {

wildcard bins even = {3’b??0};

wildcard bins odd = {3’b??1};

}

endgroup


Coverage group with event trigger

• In this case the coverage is started only when a trigger event happens. For eg you can start the coverage when the transaction ready signal is triggered from the testbench

• e.g. event transaction_ready

covergroup cp @(transaction_ready);

coverpoint ifc.cb.port; //measure coverage for port

endgroup

• In this case we start measuring coverage on the coverpoint only after the event is triggerred.


Cover group in a class

• Cover group can be defined in a module program or class.

• In any case they have to be instantiated explicitly to start measuring coverage.

• For a class you don’t need to create a separate name when you instance it. You can use the original cover group name

• Cover group is triggered whenever there are new values to the variables or expressions

• The trigger can be done by a sampling function or by using some kind of events to trigger the cover group as we had seen under coverage group with event trigger.


Functional coverage in a class

class BusTransactor; class BusTransaction

BusTransaction bt; rand bit [31:0] data; //random variables

mailbox tr_in; rand bit [3:0] port; //16 possible ports

covergroup portdata; ……………

coverpoint bt.port; ……………

endgroup endclass

function new(mailbox tr_in);

portdata = new(); // Instantiate covergroup

this.tr_in = tr_in;

endfunction

task main;

forever begin

bt = tr_in.get; // Get next transaction

ifc.cb.port <= bt.port; // Send into DUT

ifc.cb.data <= bt.data;

portdata.sample(); // Gather coverage

end

endtask

endclass


Coverage

Documents

Transcript of Coverage