There are three aspects to using metrics for Process Improvement:

  • Defining the correct metric to gather
  • Gathering the correct statistics to determine that metric
  • Making use of the results

We have already discussed the first two in previous posts so we will make the assumptions that the correct metric has been defined and the correct statistics gathered to support that metric. We now need to make use of those metrics. We will start with an example related to Software Testing and then move to the general case.

Assume we have an existing application to which we are making changes over time. It is a reasonably critical application for our company and we have built an extensive set of regression tests over the years. The regression tests have been augmented with new ones that represent implemented functionality. The regression tests are run with each release of code. We have been retaining the statistics on the number of tests that fail each time the regression tests are run. The failures are investigated, fixed if necessary (high enough priority), and the release implemented.

Recently the number of failures for each run of the regression base has been increasing and it is becoming difficult to investigate and fix all the failures prior to implementation.

The following need to be checked:

    1. Is the percentage of defects the same as before and the raw number is increasing solely as a result of the regression base getting bigger?

This requires another measurement to determine the size of the regression base so we can complete the comparison.

    1. Is the number of defects increasing as a result of the increasing size of the application?

This also requires another measurement that can determine the size of the application.

    1. Is the number of defects increasing as a result of making parts of the application active that were not used before?

Again this requires a measurement that indicates application usage.

The action to be taken depends on the answers to the above question and may vary from doing nothing to starting some refactoring and major rewrites.

We have outlined three general principles with this example:

  1. Gather the statistics (both for the activity that we want to measure and for comparison purposes)
  2. Investigate the reasons for any change
  3. Take action based on that change.

Contact us to talk about your metrics and how they can be used.