Options for a Portable Stimulus Specification Format

In our last blog post we provided some updates on the ongoing effort by Accellera to standardize “portable stimulus” in its Portable Stimulus Working Group (PSWG). We mentioned our three guiding precepts as we participate in, and help lead, this industry effort:

• Portable stimulus is not enough;  portable tests must encompass stimulus, results checking, and coverage
• Test portability must encompass both vertical reuse from IP to SoC and horizontal reuse across all verification platforms
• The tests themselves are not portable, but are generated for multiple targets from an abstract specification of the verification space

We stated our view that the goal of the portable stimulus effort can be split into three parts: defining the tests using abstract primitive operations, scheduling the tests across multiple threads and multiple processors, and randomizing the control flow to verify the full range of realistic use-case scenarios. We mentioned that the first part can be can be standardized using a simple application programming interface (API) to specify the abstract steps of the test. We have also found that the scheduling part can be handled by an expanded API. The user might want to specify the available resources and how they should be used in a particular test, for example, the number of threads running on each processor. When it comes to the third part, the randomization, an API might be feasible but there a number of candidate formats. We’d like to spend the remainder of this post examining these options.

As you would expect, we strongly believe that a graph is the best abstraction for specifying a design’s control flow. A graph-based scenario model shows how IP blocks are interconnected to form use-case scenarios and all the options available. The notion that each test can be generated by walking the graph and making randomized choices for each decision (“select node”) is easy to understand. Although some EDA vendors prefer other names, under the hood they are essentially using graphs to capture control flow and facilitate test generation. However, there is no standard for describing graphs and so our discussion involves the different ways to specify a graph and which of these specification approaches might make a good standard.

One question that we receive from time to time, even before Accellera started their effort, is whether the Unified Modeling Language (UML) can be used to specify graph-based scenario models. We have looked at this standard a number of times, and while UML diagrams can describe graph structures, the format lacks the power to specify everything needed in a scenario model. UML is primarily a visualization standard and it lacks an underlying programming model. While we see some promise for UML as a form of visual entry for graph structure, we do not believe that it is a complete solution on its own.

Since a programming model is needed, another option is to define a new programming language for graph specification. We are very wary of heading down this path. Creating a new language, verifying all its corner cases, and figuring out how to test for the compliance of its implementations are huge tasks. Even if such a graph language can be successfully specified, it adds the burden to learn a new language for every verification engineer, embedded programmer, and silicon validation team. If the new language is novel, the burden is even bigger. If the new language borrows constructs from existing languages such as SystemVerilog and C/C++, then the engineers will have to jump back and forth between two languages with just enough differences to be confusing. Either way, the risk is that the graph language will be the new Esperanto, adopted by virtually nobody.

At Breker, we originally started with a format that looked like a new language, but we quickly found that we could do almost everything we wanted in existing languages. We adopted a minimalist approach, adding only a few small extensions to C/C++ for graph specification (plus the APIs for scheduling and test abstraction). We chose to extend C/C++ since virtually every engineer learns at least some C in college. We are considering adding equivalent graph specification extensions to SystemVerilog as well. If these are identical, or as identical as they can be, engineers working in both language will not get confused. However, a SystemVerilog-only solution would never suffice, since embedded programmers and most engineers working on hardware platforms have no knowledge of testbenches at all.

Finally, we have also looked at extending the API library even further to define the graphs as well. The issue here is that that randomization requires selection from a list of alternative paths in the graph. This does not match the semantics of C/C++, SystemVerilog, and other common language. Consider, for example, as API call of “select_node(A,B)” where the control flow randomly chooses either A or B, with the other option unevaluated. This is not the way that functions or macros work and so it could be confusing to users. Thus, we have chosen to make minor extensions to existing languages to define graphs and we are urging others in the PSWG to consider this option. Stay tuned to The Breker Trekker to see how it all turns out.

Tom A.

The truth is out there … sometimes it’s in a blog.

Tags: Accellera, API, Breker, C/C++, EDA, Esperanto, functional verification, graph, graph-based, horizontal reuse, IBM, portable stimulus, PSWG, scenario model, simulation, SoC verification, subsystem, SystemVerilog, test generator, UML, Universal Verification Methodology, uvm, vertical reuse, VIP

This entry was posted on Thursday, May 7th, 2015 at 9:45 pm. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.