Recently I was asked to provide a short history of a local Quality Assurance organization.

Starting with the previous organization, I went through the 25-year history in about 5 minutes. The one major overriding change in those 25 years was the advent of the internet that created a vast information trove that is free to all. Prior to that, people were isolated within their organizations with limited contact with people with the same interests in other organizations. Solutions to problems were generally home-grown.

Having provided that very brief synopsis we were left with three questions about the purpose of the organization:

1. Are we an organization dedicated to networking?
2. Are we an education organization?
3. Are we a social organization?

Unfortunately we do not have enough of the history documented to determine what has been the best use of the time. Clearly my synopsis was based on what I remembered and I was not always in attendance at all the meetings or always on the board. So the question is still open.

If you have input on the above, you might want to consider our survey.

Photo by Jon Tyson on Unsplash

One of the problems QA people have is deciding how much proof is enough. 

Proof is expensive and time-consuming to generate.  Even with the current test tools taking screen images and storing them, you may have to annotate the proofs and combine them with other items to send to various places.   You may want to weed out items you don’t want and add things that are missed.  Having done all of that and made a nice package, you then wonder if anyone ever looks at it.   We have had clients look and some clients need it for regulatory purposes.  However, a lot want to know it is there and they could look at it if they had a need but they don’t ever open up the package and look. 

Of course it also gets superseded with new executions of the test case, new test runs or simply new code.  Sometimes the proofs accumulate in the test tool or a directory until they overwhelm storage and get put onto backup just in case they are every needed.  Eventually either they can no longer be read or someone cleans up all the old backups.

We have two major questions:

1. Are the proofs necessary?
2. How long should they be kept?

The answer to the first question depends on risk, and the ability to learn and use items from the past.  There is no doubt that higher risk items need more proof and it needs to be kept.  But the second part is even more critical, can we use the value in the proofs to inform future testing efforts in the software.  Often a second review will identify other items that might have been missed either in the existing proof or in test coverage.

The answer to the second question is actually a process that should exist for all documentation.  There is an absolute upper limit in terms of years after which, if the proof is that old and the testcase has not been re-executed in all that time, then it is not likely it is going to come back.  Alternatively, if the testcase has been executed multiple times since the the original proof was stored and all the subsequent proofs have been retained, then we will not be going back and it is probably safe to discard the older proofs.

Either way, we do not keep everything for ever.

If you have input on the above, you might want to consider our survey.

Photo by Bekir Dönmez on Unsplash