Open Source VHDL Verification Methodology/Functional Coverage
- Code Coverage (en)
- Functional Coverage (en)
The objective of functional verification is to determine if the design requirements, as defined in our specification, are functioning as intended. But how do you know if all the specified functionality was actually implemented? Furthermore, how do we know if all the specified functionality was really tested? Code coverage metrics will not help us answer these questions.
In this section, we introduce an explicit coverage metric referred to as functional coverage, which can be associated with either the design's specification or implementation coverage space. The objective of measuring functional coverage is to measure verification progress with respect to the functional requirements of the design. That is, functional coverage helps us answer the question: Have all specified functional requirements been implemented, and then exercised during simulation? The details on how to create a functional coverage model are discussed separately in the Specification to functional coverage chapter.
Benefits:
The origin of functional coverage can be traced back to the 1990's with the emergence of constrained-random simulation. Obviously, one of the value propositions of constrained-random stimulus generation is that the simulation environment can automatically generate thousands of tests that would have normally required a significant amount of manual effort to create as directed tests. However, one of the problems with constrained-random stimulus generation is that you never know exactly what functionality has been tested without the tedious effort of examining waveforms after a simulation run. Hence, functional coverage was invented as a measurement to help determine exactly what functionality a simulation regression tested without the need for visual inspection of waveforms.
Today, the adoption of functional coverage is not limited to constrained-random simulation environments. In fact, functional coverage provides an automatic means for performing requirements tracing during simulation, which is often a critical step required for DO-254 compliance checking. For example, functional coverage can be implemented with a mechanism that links to specific requirements defined in a specification. Then, after a simulation run, it is possible to automatically measure which requirements were checked by a specific directed or constrained-random test—as well as automatically determine which requirements were never tested.
Limitations:
Since functional coverage is not an implicit coverage metric, it cannot be automatically extracted. Hence, this requires the user to manually create the coverage model. From a high-level, there are two different steps involved in creating a functional coverage model that need to be considered:
1. Identify the functionality or design intent that you want to measure
2. Implementing the machinery to measure the functionality or design intent
The first step is addressed through verification planning, and the details are addressed in the section on getting from a specification to functional coverage.
The second step involves coding the machinery for each of the coverage items identified in the verification planning step (for example, coding a set of SystemVerilog covergroups for each verification objective identified in the verification plan). During the coverage model implementation phase, there are also many details that need to be considered, such as: identifying the appropriate point to trigger a measurement and defining controllability (disable/enable) aspects for the measurement. These and many other details are addressed in the detailed coverage examples.
Since the functional coverage must be manually created, there is always a risk that some functionality that was specified is missing in the coverage model.
Содержание |
Types of Functional Coverage Metrics
The functional behavior of any design, at least as observed from any interface within the verification environment, consists of both data and temporal components. Hence, from a high-level, there are two main types of functional coverage measurement we need to consider: Cover Groups' and Cover Properties.
Cover Group Modeling
With respect to functional coverage, the samping of state values within a design model or on an interface is probably the easiest to understand. We refer to this form of funcational coverage as cover group modeling. It consists of state values observed on buses, grouping of interface control signals, as well as register. The point is that the values that are being measured occur at a single explicitly or implicitly sampled point in time. SystemVerilog covergroups are part of the machinery we typically use to build the functional data coverage models, and the details are discussed in the block level design example and the discussion of the corresponding example covergroup implementations.
Cover Property Modeling
With respect to functional coverage, temporal relationships between sequences of events are probably the hardest to reason about. However, ensuring that these sequences of events are properly tested is important. We use cover property modeling to measure temporal relationships between sequences of events. Probably the most popular example of cover properties involves the handshaking sequence between control signals on a bus protocol. Other examples include power-state transition coverage associated with verifying a low-power design. Assertions and coverage properties are part of the machinery that we use to build temporal coverage models, and are addressed in the bus protocol monitor example.
Assertion Coverage
The term assertion coverage has many meanings in the industry today. For example, some people define assertion coverage as the ratio of number of assertions to RTL lines of code. However, assertion density is a more accurate term that is often used for this metric. For our discussion, we use the term assertion coverage to describe an implementation of of coverage properties using assertions.
Difference Between Cover Groups and Cover Properties
To help illustrate the difference between coverage modeled with covergroups and cover properties, lets look at a simple nonpipelined bus example, as illustrated in Figure 1.
Figure 1. Simple nonpipelined bus interface |
---|
A single write and read bus sequence for
our non-pipelined bus protocol are
illustrated in Figure 2.
Figure 2. Write and read cycles for a simple nonpipelined bus protocol |
---|
To verify our bus example, it's important
to test the boundary conditions for the
address bus for both the write sequence
and read sequence (that is, the bits
within addr at some point contained all
zeros and all ones). In addition, it's also
important that we have covered a
sufficient number of non-boundary
conditions on the address bus during our
regression. We are only interested in
sampling the address bus when the slave
is selected and the enable strobe is active
(that is, sel==1'b1 && en==1'b1).
Finally, we will want to keep track of
separate write and read events for these
coverage items to ensure that we have tested both these operations sufficiently.
This is one example of using cover groups to model functional coverage (e.g., the SystemVerilog covergroup construct). In addition, we could apply the same data coverage approach to measuring the read and write data busses.
Now, let's look at cover properties with respect to this example. There is a standard sequence that is followed for both the write and read cycle. For example, let's examine a write cycle. At clock one, since both the slave select (sel) and bus enable (en) signals are de-asserted, our bus is in an INACTIVE state. The first clock of the write sequence is called the bus START state, which the master initiates by asserting one of the slave select line (sel==1'b1). During the START state, the master places a valid address and valid data on the bus. The data transfer (referred to as the bus ACTIVE state) actually occurs when the master asserts the bus enable strobe signal (en). In our case, it is detected on the rising edge of clock three. The address, data, and control signals all remain valid throughout the ACTIVE state.
When the ACTIVE state completes, the bus enable strobe signal (en) is de-asserted by the bus master, and thus completes the current single write operation. If the master has finished transferring all data to the slave (that is, there are no more write operations), then the master de-asserts the slave select signal (sel). Otherwise, the slave select signal remains asserted, and the bus returns to the bus START state to initiate a new write operation. Multiple back-to-back write operations (without returning to the bus INACTIVE state) are known as burst write.
From a temporal coverage perspective, a set of assertions could be written to ensure proper sequencing of states on the bus. For example, the only legal bus state transitions are illustrated in Figure 3. Furthermore, it's important to test a single write and read cycle, as well as the burst read in write operation. In fact, we might want to measure the various burst write and read cycles.