Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.
Last week we discussed how Manual Testing met its Malthusian Catastrophe a long time ago. One of the proposed solutions to this issue was to Automate the testing using a variety of tools and processes. Certainly that allowed multitasking on the part of the tester. Automated tests could run on one machine ‘unattended’ while the tester concentrated on items that either could or should not be automated and new functionality that it was too early to automate. Coverage was maintained and sometimes increased.
However, unless a large effort was made to maintain the automated tests, keep them current with the changing code and conditions, and add new scripts as new functionality was added to the system the coverage either was reduced in real terms or at least relative to complete functionality of the system. There are some organizations that actually claim to have all the automation they need to properly test yet have no idea of what is in the automated suite or what it covers. There are even those for whom the possession of the automated base, whether used or not is sufficient.
We improved over manual testing. We did not solve the problem.
It is quite safe to say that Automated Testing met its Malthusian Catastrophe a while ago. Unless a very large effort is made (usually beyond the efforts of most organizations), Automated Testing is not going to cover sufficient of the code either.
Following week: Malthus on Testing – Rapid Testing
Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.