Zero Defects is an asymptote

figure: Zero Defects is an asymptote

Zero Defects

This text is from chapters 8 and 9 in booklet Optimizing the Contribution of Testing to Project Success, see booklets page, Booklet#3.
Keynote video at TestCon2018, Vilnius: How to move towards Zero Defects.

Is defect free software possible? Most people answer that defect free software is impossible. This is probably caused by lack of understanding about what defect free, or Zero Defects, really means. Think of it as an asymptote (see figure). We know that an asymptote never reaches its target. However, if we put the bar at a so called acceptable level of defects, we'll asymptotically approach that level. Only if we put the bar at zero defects, we may asymptotically approach that level.

Philip Crosby writes [ref Q5]: Conventional wisdom says that error is inevitable. As long as the performance standard requires it, then this self-fulfilling prophecy will come true. Most people will say: People are humans and humans make mistakes. And people do make mistakes, particularly those who do not become upset when they happen. Do people have a built-in defect ratio? Mistakes are caused by two factors: lack of knowledge, and lack of attention. Lack of attention is an attitude problem.

When Crosby first started to apply Zero Defects as performance standard in 1961, the error rates dropped 40% almost immediately. In my own projects I've observed similar effects.

Experience: No defects in the first two weeks of use

A QA person of a large banking and insurance company I met in a SPIN metrics working group told me that they got a new manager, who told them that from now on she expected that any software delivered to the (internal) users would run defect free for at least the first two weeks of use. He told me this as if it were a good joke. I replied that I thought he finally got a good manager, setting them a clear requirement: "No defects in the first two weeks of use." Apparently this was a target they had never contemplated before, nor achieved. Now they could focus on how to achieve defect free software, instead of counting function points and defects (which was what they had been doing until then). Remember that in accounting, being one cent off is already a capital offense, so defect free software should be a normal expectation for a bank. Why wouldn't it be for any environment?

Zero Defects is a performance standard, set by management. In Evo projects, even if management does not provide us with this standard, we'll assume it as a standard for the project, because we know that it will help us to conclude our project successfully in less time.

Attitude
As long as we are convinced that defect free software is impossible, we will keep producing defects, failing our goal. As long as we are accepting defects, we are endorsing defects. The more we talk about them, the more normal they seem. It's a self-fulfilling prophecy. It will perpetuate the problem. So, let's challenge the defect-cult, and do something about it.

From now on, we don't want to make mistakes any more. We get upset if we make one. We feel the failure. If we don't feel failure, we don't learn. Then we work to find a way not to make the mistake again. If a task is finished we don't hope it's ok, we don't think it's ok, no, we'll be sure that there are no defects, and we'll be genuinely surprised when there proves to be any defect after all.

In my experience, this attitude prevents half of the defects in the first place. Because we are humans, we can study how we operate psychologically and use this knowledge to our advantage. If we can prevent half of the defects overnight, then we have a lot of time for investing in more prevention, while still being more productive. This attitude is a crucial element of successful projects.

Experience: No more memory leaks

My first Evo project was a project where people had been working for months on software for a hand-held terminal. The developers were running in circles, adding functions they couldn't even test, because the software crashed before they arrived at their newly added function. The project was already late, and management was planning to kill the project. We got six weeks to save it.
The first goal was to get stable software. After all, adding any function if it crashes within a few minutes of operation is of little use: the product cannot be sold. I suggested the team to take away all functionality except one very basic function, and then to make it stable. The planning was to get it stable within two weeks, and only then gradually to add more functionality to get a useful product.

I still had other business to finish, so I returned to the project two weeks later. I asked the team "Is it stable?". The answer was: "We found many memory leaks, and solved them. Now it's much stabler". And they were already adding new functionality. I said: "Stop adding functionality. I want it stable, not almost stable". One week later, all memory leaks were solved, and stability was achieved. This was a bit of a weird experience for the team: the software didn't crash any more. Actually, in this system there was not even a need for dynamically allocatable memory, so the whole problem could have been avoided. But changing this architectural decision wasn't a viable option at this stage any more.

Now that the system was stable, they started adding more functions, and we got another six weeks to complete the product. I made it very clear that I didn't want to see any more memory leaks. Actually that I didn't want to see any defects. The result was that the testers suddenly found hardly any defect any more, and from now on could check the correct functioning of the device. At the end of the second phase of six weeks, the project was successfully closed. The product manager was happy with the result.

Conclusion: after I made it clear that I didn't want to see any defects, the team hardly produced any defects. The few defects found were easy to trace and repair. The change of attitude saved a lot of defects, and a lot of time. The team could spend most of their time adding new functionality instead of fixing defects. This was Zero Defects at work. Technical knowledge was not the problem to these people: once challenged, they quickly came up with tooling to analyze the problem and solve it. The attitude was what made the difference.