NVP Software Solutions

Tag: Quality Assurance

  • Malthus on Testing – Rapid Testing

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    Last week we discussed how Automated Testing has met or is going to meet its Malthusian Catastrophe fairly soon. While there is no doubt that automation has added greatly to the volume of testing that can be completed, it can be argued that the exponential increases in functionality that are easily programmed but not see easily (fully) tested will eventually overwhelm the scripts.

    Some people will advocate other methods of testing under one heading or another without explicitly mentioning the fact that everything is Risk Based. If the time or resources or both are limited, then people will use their knowledge to get as much done as possible within the available time and with the available resources. They will assess the risk and make an informed decision about what can be completed in the available time. They will assess the context in which they are working and determine what needs to be done. They will use their knowledge and experience to determine what to do. We have done this for years and there is nothing new about it. There are various techniques that can be followed to make sure that risk is reasonable. There are work processes and methodologies that will help.

    We can apply the above techniques to both are manual and automated testing and buy ourselves some more time. However, it is only a step along the way; not a solution

    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

    Next series: Communication in Testing,

  • Malthus on Testing – Automated

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    Last week we discussed how Manual Testing met its Malthusian Catastrophe a long time ago. One of the proposed solutions to this issue was to Automate the testing using a variety of tools and processes. Certainly that allowed multitasking on the part of the tester. Automated tests could run on one machine ‘unattended’ while the tester concentrated on items that either could or should not be automated and new functionality that it was too early to automate. Coverage was maintained and sometimes increased.

    However, unless a large effort was made to maintain the automated tests, keep them current with the changing code and conditions, and add new scripts as new functionality was added to the system the coverage either was reduced in real terms or at least relative to complete functionality of the system. There are some organizations that actually claim to have all the automation they need to properly test yet have no idea of what is in the automated suite or what it covers. There are even those for whom the possession of the automated base, whether used or not is sufficient.

    We improved over manual testing. We did not solve the problem.

    It is quite safe to say that Automated Testing met its Malthusian Catastrophe a while ago. Unless a very large effort is made (usually beyond the efforts of most organizations), Automated Testing is not going to cover sufficient of the code either.

    Following week: Malthus on Testing – Rapid Testing
    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

  • Malthus on Testing – Manual

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    How does this relate to software testing and in particular, for this week, manual testing? Well if we paraphrase the Mathus’ statement a little bit we could say that the ability of people to add applications is far greater than the ability of people to keep up via testing. Add to that the necessity for regression testing of the existing applications and then the interconnections with other applications that is occurring at large speed and one can see that the possibility of the application functionality and complexity far outstripping the ability of manual testers to keep up.

    A lot of people recognized this very early and proposed a variety of solutions to the problem. We will discuss those in the coming weeks with particular emphasis on Automation and Rapid Testing. Others have recognized the impossibility of keeping up and have resorted to Risk Based Testing. With very few exceptions, all testing these days is Risk Based. We accept the fact that we cannot anticipate completing everything and as a result we make a risk judgement on what is most critical to be completed. The Risk Assessment is based on a lot of assumptions and some of the assumptions are not validated thoroughly.

    It is quite safe to say that Manual Testing met its Malthusian Catastrophe a long time ago. We no longer anticipate covering even a majority of the code.

    Next week: Malthus on Testing – Automated
    Following week: Malthus on Testing – Rapid Testing
    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

  • Starting Quality Assurance – Part 4

    Starting Quality Assurance can be difficult when there are existing projects already on the go, new projects starting up and operations running at full speed. There never seems to be a good time to say “We are going to launch into Quality Assurance activities and start improving our processes”.  There will always be a more important operation or project related activity that needs to be solved and that takes precedence.

    Last week we discussed How we would want to start Quality Assurance. This week we want to look at a few examples.

    It would be nice to implement a complete Quality Assurance program using a Big Bang approach. We would decide all that needed to be done; implement all the changes and move forward. That rarely works (especially in an existing organization) and is often followed by a gnab gib (which is a Big Bang in reverse) also known as the Big Crunch. Or else we end up like the day Sweden switched the driving side.

    Since we are not dealing on a universe or country level, we can pick and choose what to do first.

    Some obvious examples include the following Quality Control Processes;

    1. Centralize defect reporting for a single project in a tool.
    2. Centralize test case management for a single project in a tool.
    3. Create standards for defect priorities and severities appropriate to your organization.
    4. Add a checkbox identifying test cases that might be needed for regression and start designating the test cases as they are being executed.
    5. Set priorities on test cases so we know which ones are important.
    6. Carry out a minimal informal Lessons Learned on a small project. No formal report is required at this stage; just people’s opinions.
    7. See what reporting could be made available without impact on the people who are testing.

    In all cases we are looking for small easy to implement changes that people can do with minimal adverse time impact.

    Next Week: Malthus on Testing

  • Starting Quality Assurance – Part 3

    Starting Quality Assurance can be difficult when there are existing projects already on the go, new projects starting up and operations running at full speed. There never seems to be a good  time to say “We are going to launch into Quality Assurance activities and start improving our processes”.  There will always be a more important operation or project related activity that needs to be solved that takes precedence.

    So, the third question we want to answer is How. Last week we discussed When we would want to start Quality Assurance and the next step is to decide How.

    The quick answer is with a minimal effort and hopefully a positive Return on Our Investment. We already discussed a somewhat similar question in our blog on Quality Assurance for Startups. The argument is similar; we need something we can do alongside everything else and yet improves our ongoing processes. The only real difference is that a Startup usually lacks any processes at all while an existing organization may have lots of processes; some of which may conflict with the new processes we want to introduce. As a matter of fact that will be the biggest impediment to this process.

    So the process differs a little:

    1. Identify the process that we want to implement.
    2. Identify the small steps needed to implement the change.
    3. Identify the impacted processes already in existence. This is the important difference.
    4. Plan around the existing processes or identify the small (again!) steps that are needed to change that process.
    5. Implement the dependent steps.
    6. Implement the new process steps.
    7. Measure the result.

    Obviously the key concern is that we may end up with too many dependent steps that stop the new process from moving forward. We could end up changing all the other processes and never get to our desired end result. That is why it is critical to have clear measurable goals in view for all the processes that are being changed.

    Next Week: Examples

  • Starting Quality Assurance – Part 2

    Starting Quality Assurance can be difficult when there are existing projects already in progress, new projects starting up and operations running at full speed. There never seems to be a good  time to say “We are going to launch into Quality Assurance activities and start improving our processes”.  There will always be a more important operation or project related activity that needs to be solved that takes precedence.

    So, the second question we want to answer is When. Last week we discussed Why we would want to start Quality Assurance and the next step is to decide When.

    The obvious answer is now but that is almost never possible. The next best answer is to set a date and an objective for another date further down for achievement of the goal. Since this is often an extra task, beyond all the tasks that are already occurring to support the normal operations of the organization; it is also critical that it be reasonably small and limited.

    The easiest and most common process that organizations start is the recording of defects in a central location using standard Severity and Priority settings.

    The subtasks are the following:

    1. Select a repository to record the defects. This could be an open source free repository downloaded from somewhere; a purchased repository or (as the last choice!) a spreadsheet.
    2. Set a goal for a project or a date for defects to have been recorded.
    3. Determine some appropriate Severities that area applicable to your business. If you can’t think of any; either look online or contact us and we will provide some standard settings.
    4. Determine some appropriate Priorities that area applicable to your business. If you can’t think of any; either look online or contact us and we will provide some standard settings.
    5. Pick a (small) project and start recording. Even if the project is part way through, starting at the midpoint is better than not starting at all.
    6. Evaluate your progress towards your goal at the set date.

    Next Week: How

  • Starting Quality Assurance

    Starting Quality Assurance can be difficult when there are existing projects already in progress, new projects starting up and operations running at full speed.  There does not seem to be a good point in time to say “We are going to launch into Quality Assurance activities and start improving our processes”.  There will always be a more important operations or project related activity that needs to be solved today that will take precedence.

    So the first question we want to answer is Why. If we cannot answer the question Why for our particular situation then it is unlikely that any initiative will succeed. Since the answer to this question has to go to the highest levels of the organisation (given the perceived cost of quality), the answer must be put in business terms.

    So, given that premise, here are some possible answers to the question Why

    1. To reduce the amount of rework in any particular process and Save Money
    2. To get a product or service to market faster and Save Money
    3. To reduce the number of defects or process problems at all stages of the development and Save Money

    In general the arguments, as is obvious from the above, are made out in terms of saving money for the organization. The above are the obvious ones that are common to almost all organizations and can be adapted to any situation.

    For your particular situation, you need to look at the processes you use or the products you produce or the way you work and ask if you can apply the above to any aspect. If you cannot apply them directly then you need to look in detail at your processes and see if anything else applies with the caveat that it must have the end result of saving money.

    Next Week: When

  • Quality Assurance for Startups – Part 4

    The last three posts have concentrated on Quality Assurance for Startups. Next week we will move into Starting Quality Assurance (for an already existing organization). However, we will finish off this week with some small examples of Quality Assurance Processes that a Startup can adopt for little or no cost.

    The first and easiest one is a Incident Tracking Process; definitions, and standards. You can easily  download a free Incident Tracker online or simply use an Excel Spreadsheet. The part that tends to get ignored is the process and standards. Generating a incident tracking flow chart can be done on paper and takes little time to create and post. Standards for the Priority and Severity for incidents can be downloaded from the Internet or generated. The resistance is usually because it is viewed as unnecessary at this stage. However, if it is created and posted at this stage, it becomes part of the culture.

    The second one is the code control and promotion process. Similar to the above, it is generally not viewed as worthwhile at the Startup stage since everyone knows where the code is and how it is to be treated. It is also likely that not all steps or stages are needed at this time. However, another flow chart showing the steps and the expectations at this stage can save a lot of concerns later. The diagram will need to be augmented as the company grows and becomes more complex but the initial diagram is quick to create and solves a lot of problems.

    The third one is testing processes. Again, it may look like too much effort for too little return at this stage but the creation of a testing process, standards, and definitions can be very effective in ensuring that everyone is working in the same way and can avoid a lot of repetition and redundancy.

    None of the above are intended to take more than a few minutes to create and set up. The payback is quite high and long term benefits are huge.