Jin Zhang, Director of Technical Marketing
Ms. Zhang has over 12 years of experience working in the Electronic Design Automation (EDA) industry, driving the effort of bringing new products to market. At Real Intent, she is the Director of Technical Marketing. Prior to that, she has worked at Cadence Design Systems and Lattice Semiconductor … More »
Economics of Verification
October 11th, 2010 by Jin Zhang, Director of Technical Marketing
In the light of the ups and downs of the world economy, it is interesting to review and see how the principles of economics work in the IC design industry, in particular with respect to verification. How much do the day to day decisions people make in doing design and verification reflect the principles of economics? In this blog, we will look at three micro economic principles and see how we can make the best choice by following these principles.
The 1st Principle – People face tradeoffs
The resources of our planet are scarce; therefore nobody can have all they want. Everybody has to face tradeoffs in making a decision. In today’s economy, the tradeoffs and choices one has to make can be particularly important.
Most managers in the IC design industry have been in a very tough situation the last few years. They have been faced with increased complexity of designs, reduced staff, tighter budgets, shortened project schedules and greater pressures from the market to perform. The choices they have to make under these constraints are challenging.
For example, at a higher level, managers may need to decide on:
At a lower level, decisions related to verification could be:
Failure to make the right choices in these decisions could potentially lead to lower quality of product, loss of profit or even bankruptcy in the current economic climate.
To best assess the tradeoffs in making these decisions, one should look at the opportunity cost involved. That brings us to principle number 2.
The 2nd Principle – The cost of something is what you give up to get it
In evaluating each choice, the rule is to see which choice has the least opportunity cost. Opportunity cost is simply what you must give up (the next best alternative) in order to get what you want. For example, you have 2 hours of free time. You could either watch a movie or take a nap. The opportunity cost for taking a nap is the enjoyment from the movie you would have otherwise had. Similarly, the opportunity cost for watching the movie is the much needed rest you would have gotten otherwise. The decision comes down to what is most important to you. It is worth noting that opportunity cost is often hard to measure and depends very much on the individual and situation involved. Nonetheless, opportunity cost is useful when evaluating the cost and benefit of choices, and the choice to go with should be the one with the least opportunity cost.
Given that verification takes 70% of the design cycle and 60% chip re-spins are due to logical/functional errors (Trends in ASIC Prototyping), it is important to invest in technology that can improve verification confidence and reduce the overall verification cycle. We will use the following hypothetical scenario to illustrate how opportunity cost comes into play in the decision making process.
The project manager at company ABC is deciding between buying more simulation licenses to do more system-level verification vs. adding automatic functional verification software to their methodology for more block-level verification. Their current verification methodology is such that limited block-level simulation is performed by designers, due to the effort involved in creating block-level testbenches. Most verification is done at the system-level by verification engineers. The company recently had a chip re-spin due to a functional error found in silicon. The project manager sees the need for more verification at both the block-level and the system-level. However, due to limited budget, they can only make investment in one area. To make the best decision, they must evaluate the verification ROI at the block-level vs. the system-level and go with the option with the most benefit, i.e. the least opportunity cost.
More and more companies are seeing the benefit of block-level verification using automatic functional verification tools. These tools operate with no testbench, therefore requiring little time and effort to setup and run. They employ formal technology to exhaustively verify the RTL blocks to catch bugs such as unreachable states, single or pair wise state deadlocks, dead codes, and synthesis pragma violations in the designs. By performing this kind of verification early in the design cycle, finding and fixing bugs becomes easier. This improves the overall quality of the RTL design before system-level verification begins, as a result, reduces the verification requirement at the system level. It is estimated that employing automatic functional verification tools can catch 50% of the design bugs early while saving 15% of the overall project cycle. This is the opportunity cost that company ABC would have to forgo if the project manager goes with more simulation at the system level.
Similarly, additional simulation at the system level could also lead to improved verification confidence. However, most things face the law of diminishing returns (also called the law of increasing opportunity cost). For example, in a production system with fixed and variable inputs (such as equipment and labor), beyond some point, each additional unit of variable input yields less and less output. The law holds for increased level of simulation. The benefit of additional simulation at the system-level is not as pronounced because significant system-level simulation is already in the current methodology. Therefore, the opportunity cost for investing in functional verification tool is less and it is where the decision should be.
With this decision made, the next question is how much verification should be done at the block level. To answer this question, we need to examine principle number 3.
The 3rd Principle – “How much” is the decision made at the margin
Some decisions in life involve either-or choices, like the one we did earlier. Some decisions involve “how much” choices, which require analysis at the margin. One needs to look at the marginal cost and marginal benefit and find the equilibrium to derive at the optimum solution. Marginal cost is the additional cost imposed when performing one more unit of an activity. Similarly, marginal benefit is the additional benefit received when performing one more unit of an activity. The point where marginal cost and marginal benefit cross is when we achieve the most efficiency.
Following our hypothetical scenario, suppose the following table shows the marginal cost and marginal benefit for an additional week of the automatic functional verification performed at the block level. It is easy to understand that fewer bugs will be found as time progresses. The “marginal benefit ($)” is calculated by the product of the number of bugs found per week and the cost to find one bug at the system level (assuming $200 in our analysis). The marginal cost is simply the salary cost to have the designer perform block-level verification. Comparing the marginal benefit and marginal cost, it is easy to see that the optimal amount of block level verification lies between week 3 and 4.
Even though we may not know these micro economics principles explicitly, most of our everyday decision makings are done by implicitly evaluating the opportunity cost and doing marginal analysis. By understanding these principles, one can form a clear framework and plug in real numbers to base the decisions upon. This is more important in our current economic condition, because a bad decision could lead to some very undesirable consequences.