Open side-bar Menu
 The Breker Trekker
Tom Anderson, VP of Marketing
Tom Anderson, VP of Marketing
Tom Anderson is vice president of Marketing for Breker Verification Systems. He previously served as Product Management Group Director for Advanced Verification Solutions at Cadence, Technical Marketing Director in the Verification Group at Synopsys and Vice President of Applications Engineering at … More »

Are Verification and Validation Different? Does Anyone Care?

 
August 25th, 2015 by Tom Anderson, VP of Marketing

For the most part, the terms “verification” and “validation” are used interchangeably in the electronics industry. However, there are many who argue that these are distinct activities in the development of SoC s and systems, performed at different times in the schedule and usually by different groups of engineers. We refer to ourselves as “The SoC Verification Company” and this is a deliberate choice we made. So we thought that it would be useful to define the two terms as we see them and talk about the similarities and  differences.

This post was inspired by an article from 2010 that our CFO and co-founder Maheen Hamid discovered recently. It opens with the “usual definitions” as follows:

  • “Validation: Are we building the right system?”
  • “Verification: Are we building the system right?”

This seems like a good place to start the discussion.

So the basic difference is that validation determines whether the system definition (specification) meets its intended purpose and usage. Verification, in contrast, determines whether an implementation of the system satisfies the definition/specification. Consider, for example, a cell phone designed for children. The validation team might check that the specified size fits comfortably in a child’s hand and that the specified video and audio quality metrics align with the capabilities of the average child.

The verification team would check during the design phase that each component complies with the specification. Of course, there is also a third team that checks during manufacturing that each instance (or representative samples) coming off the production line meets the specification as well. The article includes a very interesting Venn Diagram showing how various tasks can be grouped into verification, validation, or a combination of the two.

So let’s bear this discussion in mind as we take a look at the world of Breker and our customers. They design some of the biggest and most complex SoCs and systems in the world. A detailed specification, usually written by a combination of product marketing and system architects, is a must. Validation that this specification meets the needs of the market is primarily performed early and late in the project. Focus groups with target customers help guide the product requirements and the specification.

An architecture team typically models the design at a high level to ascertain basic functionality, estimate power consumption, and measure expected performance. This may involve running production code, which provides true validation of the hardware-software system. If the tests are hand-written, or automatically generated with Breker’s technology, one might argue that the validation covers the hardware but not the complete system.

Once a physical implementation of the design is available, for example mapped into an in-circuit emulator (ICE) or an FPGA prototype, further validation of the SoC may be possible in the context of the target system. Again, if production software is used, one can argue that the validation is more complete than with manual or generated tests. After fabrication, early systems will undergo additional validation in the lab, during Alpha test in-house, and during Beta test at customer sites.

In between, there is a whole lotta verification goin’ on. From individual IP blocks through the full SoC, simulations are run using manual or generated tests. Formal analysis and assertions may be used to verify specific functionality. Static analysis is often used to find a wide range of problems that would cause the design to vary from the specification. Some aspects of  simulation acceleration and ICE are also more about verification than validation.

By the earlier definitions and this discussion, it is clear that Breker is mostly about SoC verification, just as our company descriptor claims. Since our whole verification approach is built on realistic use cases that stress the limits of the hardware design more than production code ever would, we are arguably validating the hardware design as well as verifying it. We tend not to say this because it may confuse people who consider only full-system hardware-software validation.

Since we generate tests on “bare metal” to have more control over the hardware, we’re not directly participating in full-system validation. However, it’s not uncommon for the scenario model graphs that our customers build to call routines and sometimes even full drivers for some of the IP blocks. In this case we are providing a level of system validation by running some portions of production code within our generated test cases on the hardware design.

The bottom line is that not too many people care which word is used when, but many people care that the full range of testing and analysis spanning both terms is performed before an SoC is fabricated and the system is shipped. Do you have thoughts on this topic? Please use the comment section. We’d love to hear from you!

The truth is out there … sometimes it’s in a blog.

Tom A.

Tags: , , , , , , , , , , , , , , , ,

5 Responses to “Are Verification and Validation Different? Does Anyone Care?”

  1. Avatar Graham Bell says:

    Sales and product marketing continue to drive the spec for the design as it develops. “Feature creep” I think is an example of how validation happens between first conception and final delivery of the completed part. It is up to the design and verification teams to show it has correctly implemented.

    • Avatar Tom Anderson says:

      Indeed, it is rare for a product spec to remain unchanged throughout the project. There are many feedback loops along the way. The architecture validation phase often finds issues that result in spec changes. If a particular feature proves too expensive to implement during the design phase, this may ripple back to a spec change. Of course external factors such as competitive announcements or availability of new technology may also change the product under development. I do believe that “Validation … is primarily performed early and late in the project” but I agree with you that many of the spec changes that happen throughout the project can also be considered as part of validation. Thanks!

      Tom A.

  2. Avatar Daniel Payne says:

    I’ve always thought of Validation as confirming that my product meets the written requirements, while Verification finds any bugs in my implementation along the way.

  3. Avatar Tom Anderson says:

    Daniel, that is a nice, concise summary of the accepted definitions. Thanks!

    Tom A.

  4. Avatar Martin Vlach says:

    The difference between validation and verification is made clear, and is very important, in Quality Management and Functional Safety standards. The people who work with standards care very much which word is used. But in “classical” EDA, I agree that the words are often used interchangeably. So to me, it’s like when I’m learning a second language or a local idiom: Figure out what your customers use, and speak their language to make yourself understood.


Warning: Undefined variable $user_ID in /www/www10/htdocs/blogs/wp-content/themes/ibs_default/comments.php on line 83

You must be logged in to post a comment.




© 2024 Internet Business Systems, Inc.
670 Aberdeen Way, Milpitas, CA 95035
+1 (408) 882-6554 — Contact Us, or visit our other sites:
TechJobsCafe - Technical Jobs and Resumes EDACafe - Electronic Design Automation GISCafe - Geographical Information Services  MCADCafe - Mechanical Design and Engineering ShareCG - Share Computer Graphic (CG) Animation, 3D Art and 3D Models
  Privacy PolicyAdvertise