NVP Software Solutions

Tag: Malthus

  • Malthus on Testing – A New Paradigm

    Last week we completed our discussion of how the proposed solutions to the Malthusian Catastrophe are only postponing the inevitable. Unlike development which can always go forward and create new code, testing always has a regression component for any code that integrates with existing applications. This adds  to the testing effort but doesn’t necessarily add value.

    The proposal we outline here is not entirely new although we are carrying it to another level. Portions of this are being done and have been done for years. However, we are suggesting a more intense and more rigorous application.

    Today, we find many people integrating applications to build something. They can borrow code or entire applications and make something new from it. The statement “there is an app for that” has never been more mainstream. However, everything that is borrowed or utilized may (not always) need to be tested to some level so we can trust that it does what it says it does and operates the way we need it to in our new configuration. Add to that Security concerns dealing with applications that are not under our control and may not have been built to the same level as we need and the chance of our new application failing increases substantially.

    We are recommending that a library of code pieces are created that have the following characteristics:

    1. They carry out a known function under known conditions.
    2. They have been tested thoroughly by an independent entity to prove number 1.
    3. They are secure from being compromised.

    Number 2 above may have levels of testing just like there are levels of code today depending on the risk of the application for which they are being used but the same process applies.

    People developing applications would be able to use this library secure in the knowledge that the three points above are guaranteed.

    Next series: Communication in Testing,

  • Malthus on Testing – Rapid Testing

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    Last week we discussed how Automated Testing has met or is going to meet its Malthusian Catastrophe fairly soon. While there is no doubt that automation has added greatly to the volume of testing that can be completed, it can be argued that the exponential increases in functionality that are easily programmed but not see easily (fully) tested will eventually overwhelm the scripts.

    Some people will advocate other methods of testing under one heading or another without explicitly mentioning the fact that everything is Risk Based. If the time or resources or both are limited, then people will use their knowledge to get as much done as possible within the available time and with the available resources. They will assess the risk and make an informed decision about what can be completed in the available time. They will assess the context in which they are working and determine what needs to be done. They will use their knowledge and experience to determine what to do. We have done this for years and there is nothing new about it. There are various techniques that can be followed to make sure that risk is reasonable. There are work processes and methodologies that will help.

    We can apply the above techniques to both are manual and automated testing and buy ourselves some more time. However, it is only a step along the way; not a solution

    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

    Next series: Communication in Testing,

  • Malthus on Testing – Automated

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    Last week we discussed how Manual Testing met its Malthusian Catastrophe a long time ago. One of the proposed solutions to this issue was to Automate the testing using a variety of tools and processes. Certainly that allowed multitasking on the part of the tester. Automated tests could run on one machine ‘unattended’ while the tester concentrated on items that either could or should not be automated and new functionality that it was too early to automate. Coverage was maintained and sometimes increased.

    However, unless a large effort was made to maintain the automated tests, keep them current with the changing code and conditions, and add new scripts as new functionality was added to the system the coverage either was reduced in real terms or at least relative to complete functionality of the system. There are some organizations that actually claim to have all the automation they need to properly test yet have no idea of what is in the automated suite or what it covers. There are even those for whom the possession of the automated base, whether used or not is sufficient.

    We improved over manual testing. We did not solve the problem.

    It is quite safe to say that Automated Testing met its Malthusian Catastrophe a while ago. Unless a very large effort is made (usually beyond the efforts of most organizations), Automated Testing is not going to cover sufficient of the code either.

    Following week: Malthus on Testing – Rapid Testing
    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

  • Malthus on Testing – Manual

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    How does this relate to software testing and in particular, for this week, manual testing? Well if we paraphrase the Mathus’ statement a little bit we could say that the ability of people to add applications is far greater than the ability of people to keep up via testing. Add to that the necessity for regression testing of the existing applications and then the interconnections with other applications that is occurring at large speed and one can see that the possibility of the application functionality and complexity far outstripping the ability of manual testers to keep up.

    A lot of people recognized this very early and proposed a variety of solutions to the problem. We will discuss those in the coming weeks with particular emphasis on Automation and Rapid Testing. Others have recognized the impossibility of keeping up and have resorted to Risk Based Testing. With very few exceptions, all testing these days is Risk Based. We accept the fact that we cannot anticipate completing everything and as a result we make a risk judgement on what is most critical to be completed. The Risk Assessment is based on a lot of assumptions and some of the assumptions are not validated thoroughly.

    It is quite safe to say that Manual Testing met its Malthusian Catastrophe a long time ago. We no longer anticipate covering even a majority of the code.

    Next week: Malthus on Testing – Automated
    Following week: Malthus on Testing – Rapid Testing
    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.