Trend analysis on Defects – Part 1

One of the items that is missed frequently in most organisations is Trend Analysis on Defects. This is the trend over time and over multiple projects. We are looking for whether things are improving or not as the case may be, over time. There are a number of prerequisites which we will discuss today prior to returning to this topic in a couple of weeks with further information.

Prerequisites

1. A consistent method of classifying defects with regard to Priority and Severity. I include definitions and standards for Priority and Severity either in the Test Strategy for an organisation or in every test plan for every project. Note that without this in place; it is impossible to do any trend analysis.
2. A commitment to fairly recording defects in every phase of the project no matter what methodology you use. Otherwise it is possible to make certain phases look better than others by ignoring or failing to record the defects.
3. A commitment from management to the years it will take to obtain sufficient statistics for trends to be meaningful.
4. A commitment to and an understanding of the methodology used to reconcile projects that of different size and that are built in different ways. A very large project built in-house cannot be directly compared to a small project where the software was purchased.
5. An understanding of the statistical processes that will be used to generate the trends. It is far too easy to draw the wrong conclusion

If any of these prerequisites are not in place before anything starts, you are embarking on an expensive failure!

Photo by Stephan Henning on Unsplash