Post on 11-Apr-2016
description
SystemVerilog Verification Lecture 11
Coverage
1/28/2013 Krispan Inc. Confidential 1
Coverage types
• Coverage is a generic term used to measure the progress of verifying a design. Two types of Coverage Code and Functional
– Code coverage:
• Here you are measuring how many lines of code have been executed(line coverage).
• Which paths through the code and expressions have been executed(path coverage)
• Which states and transitions have been executed(FSM)
• Single bit variable transitions from 0 or 1 (toggle coverage)
• Most Simulators include a code coverage tool.
• Code Coverage checks how accurately your tests exercised the implementation of the design specifiacation and not the verification plan.
• 100% code coverage does not mean your design is completely verified
1/28/2013 Krispan Inc. Confidential 2
Code Coverage
• The code coverage tool will not catch mistakes you make – Suppose your design implementation is missing a feature from the
design specification. Code Coverage will not catch it.
– Suppose you forget to implement the reset logic in your flop. Code coverage will not catch it. It will report that every line in the code is exercised.
module dff ( output logic q,
input logic clk, d, reset_low);
always @(posedge clk or negedge reset_low) begin
q <= d;
end
endmodule
• The reset logic was not implemented. Code Coverage tool will not catch it.
1/28/2013 Krispan Inc. Confidential 3
TestBench with Functional Coverage
1/28/2013 Krispan Inc. Confidential 4
test
Generator
Driver
DUT
Assertions
Scoreboard checker
monitor
Test
Environment
Functional C
overa
ge
Functional Coverage
• Functional Coverage tells you how much of the design is exercised.
• If the design is exercised completely for all possible scenarios then the coverage is 100%.
• Create a test plan with the list of tests that are needed from the design specification.
• Look at the coverage results, then decide on what actions one needs take in order to achieve 100% coverage
• First action is to run the existing tests with more seeds to see if you can achieve 100% coverage.
• Next step is to write more constraints and see if that fulfils the cause.
• Write Directed tests only if absolutely necessary
1/28/2013 Krispan Inc. Confidential 5
Coverage Convergence
• For e.g. if we are looking at coverage metrics for processors
– Did we cover all instructions
– Did we cover all addressing modes
– Did we cover all valid combinations of instructions and addressing modes.
• Functional coverage identifies tested and untested areas of your design and makes sure you test for all
– Corner cases
– All states in a state machine and transitions are verified
• Use a combination of seeds, constraints and directed tests if needed to increase your coverage to 100%.
1/28/2013 Krispan Inc. Confidential 6
Functional Coverage
1/28/2013 Krispan Inc. Confidential 7
Design Specification Verification Plan
design tests Coverage
database
pass
Coverage
Analysis Debug
Functional Coverage
• You can see from the previous diagram you test for pass/fail
of a test. – Pass gives you the coverage analysis
– Fail has no significance
– Functional Coverage data is discarded if the test failed due to design bug.
– The Coverage data gives you the verification metrics and gives you a measure of how many items in the verification plan is complete which is based on the design specifications
– If the design does not match the specifications then the coverage values have no meaning.
– Analyze coverage with multiple runs with different seeds, more constraints, and directed tests if needed to achieve the coverage goal.
1/28/2013 Krispan Inc. Confidential 8
covergroup
• As we have seen before functional coverage starts with writing of the verification plan from design specification
• Then in SystemVerilog we write an executable version using covergroup and cover points
• covergroup construct is a keyword that encapsulates the specification of a coverage model. It may include a clocking event that samples values of variables and expressions which are called cover points.
• Cover groups are defined using the keyword covergroup and endgroup.
• A covergroup instance can be created using the new operator.
1/28/2013 Krispan Inc. Confidential 9
covergroup
• Coverage model definiton – Encapsulated in the covergroup construct defined by the user
– Clocking events for sampling variables or expressions which are cover points . This is done using the keyword coverpoint
– Cross coverage between coverage points using keyword cross
– Optional coverage options
• SystemVerilog functional coverage enables
– Variable and expression coverage as well as cross coverage
– Automatic and/or user-defined coverage bins
– Filtering conditions (illegal values, values to ignore, etc.)
– Automatic coverage sample triggering by events and sequences
• – Dynamic query of coverage information via built-in methods
1/28/2013 Krispan Inc. Confidential 10
covergroup
• Syntax is covergroup buslogic;
……..
……..
endgroup
buslogic bl;
bl = new;
• An instance of buslogic covergroup bl is created by assigning bl to new
1/28/2013 Krispan Inc. Confidential 11
covergroup example
program test1(busifc.TB ifc)
class BusTransaction;
//properties variables (data portion)
rand bit [31:0] data; //random variables
rand bit [3:0] port; //16 possible ports
endclass
covergroup portdata
coverpoint bt.port; // Measure coverage
endgroup
initial begin
BusTransaction bt;
portdata pd;
pd = new(); // Instantiate group
bt = new();
repeat (32) begin // Run a few cycles
assert(bt.randomize); // Create a transaction
ifc.cb.port <= bt.port; // and transmit
ifc.cb.data <= bt.data; // onto interface
pd.sample(); // Gather coverage
@ifc.cb; // Wait a cycle
end end endprogram
1/28/2013 Krispan Inc. Confidential 12
covergroup example
• In this example a random transaction is created and driven to the interface.
• Testbench samples the value of the port field using the portdata covergroup.
• 16 possible values, 32random transactions. Did your testbench have enough coverage
• vcs will generate a coverage report which gives you a summary of your coverage.
• If coverage does not satisfy your requirements then you can rerun with a new seed value, add more constraints etc.
1/28/2013 Krispan Inc. Confidential 13
Sample of vcs® output
• Coverpoint Coverage report
• CoverageGroup: portdata
• Coverpoint: bt.port
• Summary
• Coverage: 50
• Goal: 100
• Number of Expected auto-bins: 16
• Number of User Defined Bins: 0
• Number of Automatically Generated Bins: 7
• Number of User Defined Transitions: 0
• Automatically Generated Bins
• Bin # hits at least
• ================================
• auto[1] 7 1
• auto[2] 7 1
• auto[3] 1 1
• auto[4] 5 1
• auto[5] 4 1
• auto[6] 2 1
• auto[7] 6 1
• ================================
1/28/2013 Krispan Inc. Confidential 14
Sample of vcs® output
• This is just how the vcs output would look like. It is not a real output.
• You can see testbench generated the values for ports 1,2,3,4,5,6,7 but nothing was generated for 0,8,9,10,11,12,13,14,15.
• The at least option specifies how many hits are needed before a bin is covered.
• The easiest way to improve coverage is to try to generate more random transactions (more simulation cycles) or to try new seed values.
1/28/2013 Krispan Inc. Confidential 15
Coverage bins
• Bins can be created implicitly or explicitly
• When you specifiy coverpoints if you do not specify any bins then implicit bins are created.
• In the example vcs file we were trying the coverage on each port and since there are 16 ports it will automatically create 16 bins
• The number of bins to be created can be controlled by using the auto_bin_max parameter.
covergroup portdata
coverpoint bt.port;
{ option.auto_bin_max = 8; }
endgroup
• In this case total bins created will be 8 half of 16 and 1 bin for 2 ports
1/28/2013 Krispan Inc. Confidential 16
Coverage Bins
• Explicit bin creation is the preferred way. Not all cover points are of interest to the user so he can use explicit bins if he knows exactly what he wants to cover. You can also name the bins
covergroup portdata
coverpoint bt.port {
bins portzero = {0}; // 1 bin for port0
bins portone = {[1:5], 7}; // 1 bin for ports 1:5, and 7
bins porttwo = {6, [8:15]}; // 1 bin for ports 6, 8:15
}
endgroup // portdata
• In this case we created 3 bins as shown above.
1/28/2013 Krispan Inc. Confidential 17
Conditional Coverage
• You can add the iff keyword to add a condition to the coverpoint covergroup portdata
coverpoint bt.port iff(!bus.if.reset)
endgroup
• This way you can turn off cover for ports during reset assuming reset is active high.
• you can also use start and stop functions to turn on and off cover.
• Here is an example of that
1/28/2013 Krispan Inc. Confidential 18
Conditional Coverage
initial begin
BusTransaction bt;
portdata pd;
pd = new(); // Instantiate group
bt = new();
#1 pd.stop(); //stop coverage
bus.if.reset = 1; //start reset
#100 bus.ifi.reset = 0; //end reset
pd.start(); //start coverage
repeat (32) begin // Run a few cycles
assert(bt.randomize); // Create a transaction
ifc.cb.port <= bt.port; // and transmit
ifc.cb.data <= bt.data; // onto interface
pd.sample(); // Gather coverage
@ifc.cb; // Wait a cycle
end
end
1/28/2013 Krispan Inc. Confidential 19
Covergroup range
• You can use the $ to specify the upper limit if it is used to specify the right side of the range to specify the upper limit
• You can use the $ to specify the lower limit if it is used on the leftside of the range to specify the lower limit
covergroup portdata
coverpoint bt.port {
bins portzero = {[$:5]}; // 1 bin for port 0:5
bins portone = {[6,7]}; // 1 bin for ports 6 and 7
bins porttwo = {[8:$]}; // 1 bin for ports 8:15
}
endgroup // portdata
1/28/2013 Krispan Inc. Confidential 20
Creating bins for enumerated types
• For enumerated data types SystemVerilog creates a bin for each value of the enumerated type
typedef enum { idle, decode, data_transfer} fsmstate;
fsmstate cstate, nstate;
covergroup cg_fsm;
coverpoint cstate;
endgroup
• In this case it will create a bin each for idle, decode and data_transfer.
• If you want to group multiple values into a single bin then you have to define your bins just as we did before
1/28/2013 Krispan Inc. Confidential 21
Transition Coverage
• Individual State transitions can be specified for a cover point.
• For eg you can check if port went from 0 to 1 2 or 3
covergroup portdata
coverpoint bt.port {
bins p1 = (0 => 1), (0 => 2), (0 => 3); } // Measure coverage
endgroup
• Wildcard states and transitions:
– Wildcard keyword is used to specify state transitions. X,Z or ? are treated as a wildcard for 0 or 1. Here is a way to create a coverpoint with bins for even and odd values
bit [2:0] port;
covergroup cp;
coverpoint port {
wildcard bins even = {3’b??0};
wildcard bins odd = {3’b??1};
}
endgroup
1/28/2013 Krispan Inc. Confidential 22
Coverage group with event trigger
• In this case the coverage is started only when a trigger event happens. For eg you can start the coverage when the transaction ready signal is triggered from the testbench
• e.g. event transaction_ready
covergroup cp @(transaction_ready);
coverpoint ifc.cb.port; //measure coverage for port
endgroup
• In this case we start measuring coverage on the coverpoint only after the event is triggerred.
1/28/2013 Krispan Inc. Confidential 23
Cover group in a class
• Cover group can be defined in a module program or class.
• In any case they have to be instantiated explicitly to start measuring coverage.
• For a class you don’t need to create a separate name when you instance it. You can use the original cover group name
• Cover group is triggered whenever there are new values to the variables or expressions
• The trigger can be done by a sampling function or by using some kind of events to trigger the cover group as we had seen under coverage group with event trigger.
1/28/2013 Krispan Inc. Confidential 24
Functional coverage in a class
class BusTransactor; class BusTransaction
BusTransaction bt; rand bit [31:0] data; //random variables
mailbox tr_in; rand bit [3:0] port; //16 possible ports
covergroup portdata; ……………
coverpoint bt.port; ……………
endgroup endclass
function new(mailbox tr_in);
portdata = new(); // Instantiate covergroup
this.tr_in = tr_in;
endfunction
task main;
forever begin
bt = tr_in.get; // Get next transaction
ifc.cb.port <= bt.port; // Send into DUT
ifc.cb.data <= bt.data;
portdata.sample(); // Gather coverage
end
endtask
endclass
1/28/2013 Krispan Inc. Confidential 25