Open side-bar Menu
Donald Cramb, Director of the Consulting Services Division, EVE
Donald Cramb, Director of the Consulting Services Division, EVE
Donald Cramb is director of the Consulting Services Division of EVE-USA in San Jose, Calif., and is responsible for customer services, applications and design solutions to support specific customer requests. Previously, he was a partner at ArchSilc Design Automation, a company focused on … More »

The Never-ending Evaluation

March 7th, 2012 by Donald Cramb, Director of the Consulting Services Division, EVE

How long did your last EDA tool evaluation take? One month? Three? Six? The EDA industry seems to be the land of the never-ending evaluation.

Of course, it’s totally understandable. EDA tools are amazingly complex, and thorough evaluation ensures that you are getting the right tool for the job.

Evaluations also come at a cost. There’s a direct cost to the evaluator, in that evaluations require internal resources that could otherwise be applied to a live project. There’s also an indirect cost to the industry as a whole, as EDA vendors have to loop the cost of evaluations back into the price of their products. Thus, it’s in everybody’s best interests to maximize the efficiency of evaluations.

Let’s take a look at some of the reasons why EDA evaluations can take forever.

Part 1: Deciding to Evaluate
The length of an evaluation is influenced by factors that occur while you are just deciding to evaluate.

Unrealistic Expectations
No vendor likes to turn down sales opportunities, and sometimes evaluations are initiated even though there is a low probability of success. Evaluations with unrealistic or unachievable expectations can be a huge waste of time and energy for both parties, so do your own research and make sure the product in question is appropriate for your application.

No Management Buy-in
Engineers have different priorities than upper management. If an evaluation begins without the proper buy-in, both the evaluator and the vendor could be in for a rude awakening later in the process. Evaluations can be delayed, restarted, or cancelled outright by upper management, so make sure you have the required approval and budget before you begin.

Globalization vs. Centralization
Global design teams and centralized CAD organizations can add complexity and delay to the evaluation process. Talk to your CAD team and to other groups that will be part of the decision making early to determine who will conduct the evaluation, and what will be tested. Your priorities may not align with another group’s.

Part 2: Preparing For the Evaluation
Once you have decided to move forward with the evaluation, there is still a lot of prep work to be done. Managing these issues before you start the evaluation clock will save time and effort further down the road.

Not Having a Plan
Every successful evaluation requires a plan. Without a detailed evaluation plan, EDA vendors can easily waste days or even weeks focused on the wrong priorities. Work with your vendor to create a prioritized list of criteria with measurable metrics. Plans may change over time, but it’s important for both sides to start the evaluation with the same global priorities and expectations.

IT and Security Logistics
EDA evaluations are often performed onsite or using secure network access, and logistics like getting a contractor badge and VPN account, installing new tools and licenses, and even procuring desk space can cause lengthy delays in an evaluation. Make your IT and security arrangements early so the FAE can hit the ground running.

Unstable/Incomplete Testcases
Using an unstable or live testcase, or one without a golden reference, often extends the length of an evaluation. Many cycles are lost due to missing or frequently changing code, or searching for a problem with the tool when the issue is really in the testcase. It’s best to use a recent, previously verified testcase for your EDA evaluations.

Part 3: During the Evaluation
Unexpected problems and changes will occur during the course of an evaluation—it’s a fact. How these problems are managed will determine their impact on the length of the evaluation.

No Active Support for the Vendor
EDA FAEs are skilled at deploying solutions across a wide range of applications, but they do not know the intimate details of your design. Many problems encountered during the evaluation can be easily solved by someone more familiar with the design. Make sure that you have a resource available to support the vendor during the evaluation.

Sticking with the Original Plan
As priorities and schedules change, it is critical to update your evaluation plan to keep your EDA vendor on track. The evaluation plan should be a live document that gets continuously reviewed and tuned over the course of the evaluation.

Lack of Communication
The biggest source of delays in EDA evaluations stem from a lack of communication.

Evaluations are dynamic projects and changes are unavoidable, but the frequency and openness of your communication will determine how much time is lost. Schedule regular progress meetings, and don’t play your cards too close to your chest. The best way to get what you want out to the evaluation is to let the vendor know exactly what you’re looking for.

Developing a plan with achievable goals, a reasonable timeline and measureable metrics will help to guarantee the success of an EVE tool evaluation. Planning ahead will save valuable resources for both the EDA vendor and your company.

Related posts:

2 Responses to “The Never-ending Evaluation”

  1. Rob Garneau says:

    We have been teaching high tech companies around the world for years how to better plan for and manage evaluations. Your article is excellent and I thought I would add some additional points.

    As you mentioned in your Unrealistic Expectations paragraph, vendors don’t want to turn down business–even in cases where it might not make sense. More specifically, Sales People don’t want to turn down business. The Sales Person has their quota to reach and they may not be the most judicious user of the AEs’ time. If the Sales Person can get resources to perform evaluations on their accounts, many believe it improves their ability to make sales. So, the sales incentives may promote evaluations and lengthy/costly sales cycles.

    Very often there is no attempt to make the sale without an evaluation. As you stated, there is a cost to the customer of the resources they must allocate to evaluate a solution. In addition, there is another cost to the customer in the delay of implementing the value of the new solution on their production products. This should be communicated to the customer. Too often evaluations are entered into without any forethought or attempt to consider other alternatives to accelerate the sale for the benefit of both parties.

    Alternatives to hastily entering into an evaluation might be to utilize references, success stories, a focused demo, industry articles, etc.

    It is for that reason the AE is ultimately responsible for determining how to best spend their time between the various sales people that are vying for it. We enable the AEs to understand when it makes sense to engage and give them tools to work with the sales person to explore alternatives before hastily entering into a costly and lengthy evaluation.

    When an evaluation makes sense, it is paramount to put together a plan in advance and get buy-in from your customer and the decision maker. The plan does not have to be overly extensive but should include important content such as the completion date, who’s involved (from customer and vendor), what design/project will be the test case, what are the milestones (e.g. installation, training, review meetings), and the criteria for success.

    If you are interested in discussing this topic further or learning more about the sales training we offer to enable AEs to increase business results, feel free to contact us at

  2. Randy Thompson says:

    The biggest thing you need to insure a successful evaluation is a set of success criteria. Without this, the end of the evaluation keeps moving and results in lots of wasted resources and an unhappy outcome. The success criteria is also key to making the qualification decision. I.e., For the AE, is this an evaluation that our product can win?

    Great article!

Leave a Reply

Your email address will not be published. Required fields are marked *


DownStream: Solutions for Post Processing PCB Designs
TrueCircuits: IoTPLL

Internet Business Systems © 2018 Internet Business Systems, Inc.
25 North 14th Steet, Suite 710, San Jose, CA 95112
+1 (408) 882-6554 — Contact Us, or visit our other sites:
TechJobsCafe - Technical Jobs and Resumes EDACafe - Electronic Design Automation GISCafe - Geographical Information Services  MCADCafe - Mechanical Design and Engineering ShareCG - Share Computer Graphic (CG) Animation, 3D Art and 3D Models
  Privacy PolicyAdvertise