Colin Walls has over thirty years experience in the electronics industry, largely dedicated to embedded software. A frequent presenter at conferences and seminars and author of numerous technical articles and two books on embedded software, Colin is an embedded software technologist with Mentor … More »
Product quality: belief or proof?
October 15th, 2014 by Colin Walls
There are two aspects of any product that give me great pleasure: good design and quality of manufacture. This applies to just about anything: a pair of shoes, a car, a house, a piece of software … Assessing both these parameters can be hard. A pair of shoes must look good, be comfortable to wear and last well; this takes time to evaluate. A car needs to be enjoyable to ride in, drive well and offer long term reliability and economy; again, not something to be measured quickly. A house can be even more difficult, as you need to live in it through a full year of seasons at least and, even then, you may still be evolving the way that you use the space, which will establish how good the original design was.
In many ways, software can be the most difficult product of all to assess, as programs represent the most complex “machines” mankind has ever produced …
To explain this last point, the most complex mechanical device ever created is the Space Shuttle orbiter, which has around a million moving parts. In software terms, I suppose a bit is a moving part and program code consisting of 100m bits is not unusual. It is, therefore, unsurprising that assessing the design of a program and figuring out how well it was implemented is a big challenge.
How well a program is designed can often be determined by just using it. Fortunately many software vendors offer evaluations to enable potential users to try before they buy. On countless occasions, I have downloaded and installed a promising program, only to uninstall it after only a few minutes use, during which I concluded it was not for me.
Quality is more difficult. To some extent, you need to rely upon the vendor’s claims and reputation. For embedded software products, the possibilities for quality control and detailed testing are quite variable:
The ways that vendors portray quality in embedded software products is interesting. Some companies use beta testing programs to get their customers to find bugs. This is not a recommended way to build a lasting good reputation.
Other companies are more creative. One of the first RTOS products on the market was VRTX from Hunter & Ready [later Ready Systems, who were acquired by Microtec Research, who were acquired by Mentor Graphics …]. They were very confident about their product being bug free. They ran an advertising campaign called “A Bug for a Bug”, where they offered a VW Beetle car as a reward for finding a bug in the kernel. I have no idea whether there were lots of strings attached or if they ever gave away a car, but it was a very memorable program.
When you are buying a software product, do not be shy about querying the quality assurance and testing procedures in use. A salesman saying that he believes in his product is good, but not good enough; I want proof!
Years back, there was a compiler vendor [I will not name them – they were later acquired by a semiconductor company, so never heard from again] who offered quite good quality documentation. At the bottom of many pages were Latin phrases praising the glory of God. I have total respect for other people’s religious beliefs, even if I do not share them. But this is not the business of software manuals. We always used to say “They have faith; we have Technical Support.”