Visibility Enhancement for Full-Chip Simulation Methodology Backgrounder
Executive Summary
Software simulation is the primary method used to verify logic in integrated circuit (IC) designs. When simulation discovers logic errors through unexpected logical behavior, users must trace the causes. This tracing requires that logic values be recorded during simulation. For large designs, recording these values adds enormous overhead to the simulation process. This causes users to adopt a variety of costly strategies for reducing this overhead. New Visibility Enhancement technology enables methodologies that reduce the need for this manual effort and improves the overall productivity of logic verification with simulation. This technology identifies a minimal subset of signals required for full visibility and automatically expands the data for other signals from this recorded subset during debug.
Full-Chip Simulation, Visibility, and Debug
Software-based simulation is the primary method used to verify IC designs. According to Gartner-Dataquest, over 160,000 simulation licenses are in use for verifying ASIC and FPGA designs [i]. Instrumental to this widespread use is a vast investment in user-created scripts and regression suites that leverage the flexibility of the venerable simulator. Continued performance and capacity improvements in simulation technology, as well as new functionality including assertions and high-level abstractions such as transactions, have extended the usefulness of simulation and preserved the infrastructure investment. As the size of ICs has exploded, the time required to thoroughly verify them with simulation has risen dramatically. This increase is especially acute when the values of the signals must be recorded in order to facilitate visibility into understanding the design's behavior.
Understanding design behavior is one of the most difficult challenges in IC verification. The process of understanding the behavior of a design is known as "debug" or "debugging" and is an interactive process separate from simulation [ii]. Debug requires recording the signal values generated during simulation. High signal visibility is critical to effective debugging.
Design complexity, as measured by the number of signal value combinations, grows faster than design gate counts Consider a design that currently contains N gates, the next generation will contain 2N gates, the lower bound number of signals will be 2N, and the corresponding minimum number of signal value combinations will be 2(2N) or 4x more than the current design (Figure 1). With the doubling of design sizes every 24 months [iii] continues, the overhead of gathering the signal data required for visibility will continue to increase significantly.
Figure 1 - Signal combinations grow exponentially with the number of signals in each new design generation
In order to address this challenge, simulators have the means to provide visibility of signal values over a period of time. Commercial simulators provide this capability either natively or via plug-ins that are called through language-specific interfaces such as Programming Language Interface (PLI) for Verilog and VHDL Procedural Interface (VHPI) for VHDL. To accommodate thorough understanding of design behavior, visibility functions dump the value and time of each signal change to an external file (hence the functions are often called "dumpers"). Some common formats for these external files include Verilog Change Dump (VCD) and the Novas Software Fast Signal Database (FSDB), both of which can be imported into popular debug systems.
To minimize simulation run time impact, dumpers are specifically optimized for performance. In addition, most simulators compile the functions with additional optimizations through streamlining techniques. As a result, it is feasible to dump data for all of the signals in the early block-level development stage. However, in later stages, such as the sub-system design stage, the performance impact on simulation from full-signal dumping becomes noticeable. At the full-chip stage, the performance impact from dumping data for all signals in the design is significant (Figure 2). Customer data shows dumping data for all signals during a typical full-chip simulation results in about a 3x to 4x slow down in run time as compared to no dumping [i].
Figure 2 -The impact of full signal visibility with the progression to each development stage
Full dumping of signals for designs containing millions of gates also requires a massive amount of disk storage. The architecture of the FSDB format offers, on average, 10 times greater compaction than the VCD format. Still, it is not uncommon for a design between 10 to 20 million gates to require hundreds of gigabytes of disk storage for a suite of long simulation runs.
Current Full-Chip Simulation and Visibility Practices
Simulation of most designs is a multi-step process that begins at the block level, migrates to the sub-system level, then moves to full-chip simulation. The purpose of the full-chip simulation is to both verify that the design contains no bugs, especially in the block interfaces and system-level interaction, and provide relevant data if a bug is detected. This data is usually the dumped signal values which can, through involved manual interaction, identify the root causes of the bugs. However, as a result of the potential negative impacts on simulation performance and disk storage, many verification teams have modified their dumping methodology by eliminating full-signal dumping during most of the simulation runs.
These alternative methodologies may overcome simulation issues associated with full- signal dumping, but in many cases they also introduce impediments to signal visibility. Lack of signal visibility, in turn, slows down debug efforts. This is especially problematic with the introduction of new root-cause analysis capabilities that often require much signal data. As a result, while these methodologies may speed up simulation, they potentially result in longer overall verification times when debug is required. Several of these alternative methodologies are described in the following sections.
Short Tests
One common methodology is to limit the simulation time of each run usually by designing the test itself to be short. Several short run-time simulations effectively replace one long, sequential simulation. The advantage of this approach is that it allows parallelization of the simulation across an array of compute servers (simulation farm). Assuming that all the runs can be parallelized, the time savings will be based on the longest run and computed as follows:
N= time of original simulation with full dumping
ni= time of ith simulation with full dumping
The time savings is: N- max{ v1 , …, v1}
The shorter the "longest" simulation run is, the larger the savings.
There are several disadvantages of this approach. Primarily, the task of dividing the original run is usually complicated. The starting conditions of any given run are dependent on the terminal conditions of the previous run in the sequence. As a result, the entire simulation is usually run once to provide snapshots of the entire state for initialization of each of the short runs. Subsequent changes to the design usually require repeating this process to determine the initialization values. In reality, many verification teams assume initialization values rather than recalculate them. By assuming conditions, several design structures such as reset control and FIFOs are not thoroughly tested. The other notable disadvantage is that this approach does not address the large disk storage requirements for the dumped signals. The total size of the files from all the runs is approximately the same as the file size from the original run.
Partial Dumping - Signal-based
Instead of restricting the simulation time, another common methodology is to limit the amount of dumped signal data by selecting a subset of signals a priori. Since this methodology somewhat relies on guessing which signals are important and necessary for debug, it inevitably leads to multiple, lengthy simulation iterations when the root cause of a bug cannot be immediately determined. The impact on actual time spent is not readily determinable since the number of iterative simulation runs required to dump out sufficient signal data is not known in advance. While this method has merits in preventing performance degradation and reducing file size, it does not accommodate a predictable debug flow.
Accounting for the likelihood that more than one simulation may be required, the time savings is computed as follows:
N= time of entire simulation with full dumping
ni= time of entire ith simulation with partial dumping
P(Ai)= the probability that the ith simulation detected a bug without enough dumped data
The time savings is: N-( v1 + P(A1)v2 + … + P(Av-1)v1)
The larger the savings will be the result when the likelihood that bugs are detected is lower and the likelihood that enough data is dumped for debug is higher. Note that the savings could be negative if too many iterations occur.
Partial Dumping - Time-based
Time-based partial dumping methodology limits the amount of dumped signal data based on time or events. This methodology, in part, also relies on guessing the relevant times for dumping signals that are needed for debug. More effort is required to implement this methodology since the time scope or event must be described in the test bench using conditional statements. Multiple simulation iterations are very possible since the root cause of a design behavior problem cannot be immediately determined in advance. This methodology may limit performance degradation and reduce file size, however, it does result in an unpredictable simulation and debug flow.
Accounting for the probability that more than one simulation is required, the time savings is computed as follows:
N= time of entire simulation with full dumping
ni= time of entire ith simulation with partial dumping
P(A1)= the probability that the ith simulation detected a bug without enough dumped data
The time savings is: N-( v1 + P(A1)v2 + … + P(Av-1)v1)
Here again, lower likelihood that bugs are detected and higher likelihood that enough data is dumped for debug result in the larger savings. Note that the savings could be negative if too many iterations occur.
No Signal Dumping
According to Novas user survey data, approximately one-third of verification teams use a methodology of first simulating with no signal dumping and then simulating a second time with full signal dumping if a bug is detected. Generally, the hope is that the first simulation run does not detect a bug, thus avoiding the need for a second run. In this no-bug scenario, the highest simulation performance is achieved and no extra disk storage is required. However, when bugs are detected, the second simulation run negates these advantages. Considering that the second run is three to four times longer (on average) compared to the original run, the total time from start of the first simulation to obtaining dumped signal values is about 4 to 5 times the original run time.
Accounting for the probability that the second simulation may be required, the time savings is computed as follows:
N= time of entire simulation with full dumping
n1= time of entire 1th simulation with without dumping
P(A1)= the probability that the 1th simulation detected a bug
The time savings is: N-( v1 + P(A1)N)
Savings will only occur if there are no bugs in the first run.
Disadvantages of Workarounds
The goal of the methodologies described above is to speed simulation time and minimize dump file sizes. In this scenario, signal visibility becomes a secondary priority, and complete debug is often delayed. Since debug accounts for 50 percent of the overall verification time2, these methodologies may not optimize the higher-level goal of faster verification. When bugs are detected by simulation, visibility via dumped signal values is necessary in order to proceed with debugging. Therefore, a simulation methodology can be described as delivering "faster verification" only if it decreases the total time to:
- Determine if a snapshot of the design does or does not contain bugs
- Produce data for debug if bugs are detected
- Find the root causes of bugs
Without new technology or methodologies, verification teams face the choice of 1) fast simulations without any signal visibility, 2) extremely slow simulations with high signal visibility, or 3) an unpredictable number of simulation iterations with partial signal visibility (see Figure 3). Empirically, simulation performance appears to be indirectly proportional to signal visibility. Clearly, verification teams need fast simulation with high visibility to accommodate efficient debug.
Figure 3 - Tradeoff of visibility vs. impact using different dumping methods
Visibility Enhancement Technology
Novas Software has pioneered new "visibility enhancement"4 technology that provides verification teams with an optimal trade-off between simulation performance and signal visibility. Visibility enhancement technology improves verification methodologies by enabling an analysis-driven approach to partial signal dumping that limits the impact on simulation performance while still providing full signal visibility for debug. For simulation, the technology contains two primary components: design analysis to determine an optimal subset of signals to dump and data expansion to automatically regenerate missing signal data during debug. This technology is delivered in the Novas Siloti™ SimVE™ product.
The first step in the visibility enhancement flow is to run the analysis in order to generate a small list of signals that must be made visible through dumping. Next, this list of signals is uploaded to the simulator so it can dump out the associated values at run-time. Because this list usually is significantly smaller than the full set of signals in the design, the simulation runs much faster than a full-dump simulation. Simulation benchmarks show that this type of partial-signal dumping typically increases simulation run time by only 20 percent compared to simulation with no-signal dumping. Furthermore, the same benchmarks show that partial-signal dumping reduces dump file sizes by more than 80 percent as compared to full-signal dumping.
After simulation is complete and the partial dump created, the next step is to invoke the data expansion capability and the Novas Verdi™ Automated Debug System. As the user views signal data and analyzes the operation of the design within the Verdi environment, the Siloti data expansion engine generates the missing signal values and transparently passes them to the Verdi debug system. The high performance of data expansion is attributed to dynamically generating only those signal values required by the current debug operation. By expanding the limited amount of dumped signal data for debug on-demand, visibility enhancement technology effectively increases signal visibility toward 100 percent. Despite the fact that only a small amount of data was dumped during simulation, there are no negative effects on the debug process.
Visibility Enhancement Analysis
Visibility enhancement requires that the register transfer level (RTL) source code or gate-level netlist be fully analyzed so that the functionality of the design is understood. This analysis includes understanding the clock synchronization and corresponding levelized signals. Next, the design is analyzed to determine the minimum and sufficient set of signals (referred to as "essential" signals) that should be dumped in order to provide visibility of all the design signals. As shown later, the resulting set of signals is a small subset of all the signals and depends on the efficiency of the specific visibility enhancement technology (the better the technology the smaller the set of signals). On average, data gathered from designs using today's state-of-the-art technology shows that the analysis usually results in a selection of 15 to 20 percent of the design signals that later provide close to 100 percent visibility using data expansion technology.
If the verification team wants to debug only a specific block or sub-system rather than the entire design, then the scope of the analysis can be scoped as such. Specifying a subset of signals may provide additional simulation performance and dump file size optimization. The output of this process is an optimal list of signals to dump that will provide close to 100 percent post-simulation visibility for the desired blocks or sub-systems.
Data Expansion
Given that the simulator produces only a partial dump file based on the essential signals, the remaining signals must be generated for the debug system. The data expansion engine imports the dump files along with the RTL or netlist. By understanding the functionality of the design, the engine is able to take the known signal values and generate missing signal values based on an efficient series of calculations. These cycle-accurate signal values, providing close to 100 percent visibility, are made available to the debug system.
To ensure visibility enhancement delivers optimal performance, the data expansion engine dynamically communicates with the debug system. During debug, the engineer is typically only interested is a small set of signals at any one time. For example, the engineer may want to trace a cone of combinational logic to the fan-in register points during the previous clock cycle. In this case, the debug system only requests the values for this set of signals. In response, the data expansion only provides the corresponding values in an "on-demand" manner. The engine calculates the requested values for the signals of interest during the specified time window. This on-demand approach saves time by preventing calculations of signals values that are not needed for the current debug task. The corresponding time required for data expansion is usually small enough that it is not noticeable to the user, making the debug session look and feel like it is operating on a full-signal dump.
Visibility Enhanced Methodologies
Visibility enhancement technology enables the methodology of "always dumping" during simulation and can also be applied to existing signal dumping methodologies. In either case, visibility enhanced methodologies reduce the simulation run time and use of disk space.
Always-dump Methodology
Ideally, simulation verifies the design with the fastest performance possible while providing full-signal visibility. As stated earlier, the common means to obtain visibility is through dumping, which slows down simulation. The goal with the visibility enhanced "always-dump" methodology is to maximize the trade-off between simulation performance and signal dumping, while providing full-signal visibility.
The first step to enable this methodology is to run visibility analysis on the entire design. Next, the resulting list of essential signals is uploaded into the simulator. During run-time, the simulator produces a "limited" dump file which is much smaller as compared to a "full" dump file. No further simulation is necessary since all signal values throughout all the simulation time are visible using data expansion. The time savings is computed as follows:
N = time of entire simulation with full dumping
E = time of entire simulation with without dumping
x = the % impact on run time using the visibility enhancement methodology
The time savings is: N- xE
Savings will always occur and be greatest when x is minimized. Benchmark data from Novas shows that x is approximately between 15 to 25 percent for many commercial simulators.
If this methodology is adapted for a suite of simulation regression tests, file management becomes a notable, but manageable issue. Many regression environments are set to run anywhere from ten to thousands of simulations in an unattended batch mode. Usually monitors in the test environment indicate if an error occurred and which test produced it. Using the visibility enhanced always-dump methodology ensures that the verification team can immediately begin to debug if a test fails due to an error. However, without any file management, the number of files with dumped signals could be enormous.
In order to conserve disk space, the dump files can be deleted for individual tests that pass without errors. This step can be implemented by either modifying existing regression scripts or running a separate script after the completion of each test. Using this approach solves issues regarding the large number of files. Below is sample script that implements this task::
# script to retain the FSDB dump file only if an bug exists
#!/usr/local/bin/perl -w
$argc = @ARGV;
if ($argc == 2) {
$sim_log = $ARGV[1]; # points to the simulation log file
$fsdb_file = $ARGV[2]; # points to the dumped FSDB file
} else { # exit - wrong number of arguments
print "Usage: $0 simulation.log test.fsdb n";
exit -1;
}
if (!-e $sim_log) { # check if log file exists
warn ("nWARNING:****** $sim_log does not existn");
die ("Please specify a correct simulation log file name.nn");
}
if (!-e $fsdb_file) { # check if FSDB file exists
warn ("nWARNING:****** $fsdb_file does not existn");
die ("Please specify a correct simulation log file name.nn");
}
open(SIMULATION_LOG, "$sim_log");
while ($line = <SIMULATION_LOG>) {
if($line =~ /error/)||($line =~ /ERROR/ ||($line =~ /Error/) {
print ("Error detected ... ");
print ("Retaining the FSDB file $fsdb_file - required for debug.");
close (SIMULATION_LOG);
exit 0;
} else {
print ("No error detected.");
print ("Deleting the FSDB file $fsdb_file - not required for debug");
system("rm $fsdb_file");
print "Done!n";
close (SIMULATION_LOG);
exit 0;
}
}
At first glance, the benefits of the visibility-enhanced always-dump methodology may not be readily apparent. Indeed, the simulation run times are longer compared to simulating without dumping. This comparison makes the visibility enhanced always-dump methodology seem longer than the methodology of first simulating without signal dumping. However, errors are almost always detected during this first simulation, especially in the beginning of the full-chip simulation phase. As a result, the traditional approach usually requires a second simulation with full dumping for visibility. While this approach requires two simulation runs, the visibility-enhanced methodology only requires one. By eliminating the second run, the visibility-enhanced methodology results in a shorter overall time, around 3-4x, when measured from the start of the simulation to the start of debug. (see Figure 4)
When compared to the partial-dump methodology, the visibility-enhanced always-dump methodology also decreases the overall time, although it is dependent on the specific design, amount of dumping, as well as other factors. Unlike the partial-dump methodolog, there is no need to add complex code to the verification environment to specify what and when to dump. As a result, the simplicity of the visibility-enhanced always-dump methodology raises productivity.
Figure 4 - Comparison Dump Methodologies (based on typical run times)
Existing Existing
Visibility enhancement can be immediately incorporated into existing methodologies - short tests and partial dumping. Only a few preliminary steps need to be performed upfront. The enhanced methodology will avoid dumping of all signals within a specified scope and/or time window, and instead, dump only the corresponding essential signals.
The first step is to run visibility enhancement analysis on the scope of interest. In the case of the short test methodology, the scope is the entire design whereas the scope for the partial-dumping methodology will be a specific module or modules of the design. The result of the analysis is the list of essential signals that should be used instead of the original dump list. The quantitative benefits of using visibility enhancement will vary according to the methodology and are summarized the following section.
Results
To demonstrate the benefits, three different methodologies were applied to a 20-million gate-level Verilog design: a) simulation with no dump, followed by a simulation with full dump; b) simulation with full dump; and c) visibility-enhanced (essential signals only) dump. For all methodologies, the same testbench stimulus was utilized. The study involved a total of four cases, each using a different testbench. (see Table 2).
|
Methodology
|
Case A
Run Time (minutes) |
Case B
Run Time (minutes) |
Case C
Run Time (minutes) |
Case D
Run Time (minutes) |
|
No dump, then full dump
|
813
|
908
|
2640
|
531
|
|
Full dump
|
670
|
716
|
2151
|
399
|
|
VE dump
|
208
|
295
|
546
|
149
|
Table 2 - Run time results for different methodologies
The corresponding dump files (in the FSDB format) were also recorded as shown below (see Table 3).
|
Methodology
|
Case A
FSDB Size (MB) |
Case B
FSDB Size (MB) |
Case C
FSDB Size (MB) |
Case D
FSDB Size (MB) |
|
Full dump
|
2500
|
3700
|
2094
|
2100
|
|
VE dump
|
220
|
323
|
283
|
61
|
Table 3 - Dump file size results for corresponding methodologies
The run-time savings using the visibility-enhanced dump was between 58 and 75 percent compared to full dumping, and between 67 and 80 percent compared against the no-dump-then-full-dump approach. The resulting file size savings was between 86 and 98 percent for the four cases.
Summary
Visibility enhancement techniques allow an optimal tradeoff between performance and signal visibility for simulation of large designs and the sub-systems of large designs. These techniques provide signal visibility by leveraging a small amount of dumped signal data to eliminate simulation iterations and provide a predictable simulation flow with full signal visibility. As compared to traditional regression methodologies, the dumped file sizes are significantly smaller as is the overall simulation run time. When deployed across many existing regression suites, visibility enhancement enables optimal simulation, efficient debug, and tremendous overall resource and time savings.
[i] Gartner-Dataquest email communication to Novas Software, Inc., September 2006
[ii] "Novas Debug Technology White Paper", Novas Software, Inc.
[iii] International Technology Roadmap for Semiconductors, http://public.itrs.net