NVP Software Solutions

Tag: Quality Control

  • Malthus on Testing – Manual

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    How does this relate to software testing and in particular, for this week, manual testing? Well if we paraphrase the Mathus’ statement a little bit we could say that the ability of people to add applications is far greater than the ability of people to keep up via testing. Add to that the necessity for regression testing of the existing applications and then the interconnections with other applications that is occurring at large speed and one can see that the possibility of the application functionality and complexity far outstripping the ability of manual testers to keep up.

    A lot of people recognized this very early and proposed a variety of solutions to the problem. We will discuss those in the coming weeks with particular emphasis on Automation and Rapid Testing. Others have recognized the impossibility of keeping up and have resorted to Risk Based Testing. With very few exceptions, all testing these days is Risk Based. We accept the fact that we cannot anticipate completing everything and as a result we make a risk judgement on what is most critical to be completed. The Risk Assessment is based on a lot of assumptions and some of the assumptions are not validated thoroughly.

    It is quite safe to say that Manual Testing met its Malthusian Catastrophe a long time ago. We no longer anticipate covering even a majority of the code.

    Next week: Malthus on Testing – Automated
    Following week: Malthus on Testing – Rapid Testing
    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

  • Quality Assurance for Startups – Part 2

    In the last blog we discussed why Quality Assurance for Startups was appropriate despite it not being the first thing on startup leaders’ minds. This is understandable given the amount of other work that needs to be done just to get going as a new business.

    It would be pointless to introduce a full Starting-Quality-Assurance process in the beginning stages. The effort involved would detract from the other to-dos and could interfere with growth. With that said, we can introduce a limited Quality Assurance program during the startup stage that will benefit all without adversely impacting the growth of the organisation as follows:

    1. At the end of each week, take 15 minutes and review what has been done and how it is being done.
    2. At the end of each month, take 1 hour and review what has been done and how it is being done.
    3. In both cases, take brief notes and file them.
    4. At the completion of the one month meeting, consolidate the notes and divide them into groupings. For example: Development; Quality Control; Process; Customer Feedback: Recurrent Issues
    5. Look for items that might be costing you money; that are easy to correct and can be implemented immediately.
    6. Look for items that have a medium term to correct and may still be worthwhile
    7. The biggest items to look for are those that may not be easy or fast to correct but may have a big impact in the future if left unchanged.

    If the items fall into the easy category; make the changes now and implement going forward.

    If the items fall into the medium category; try to subdivide them into things that can be done immediately and implement some of those.

    If the items fall into the long-term or hard category; there may be a need to assess how much effort it will require to implement a change.

    We are going to tackle how to implement next week.

  • Quality Assurance – Implementing Right the First Time

    The last three NVP Blogs have discussed Doing it Right the First Time. The last one addresses the issue of Implementing Right the First Time. This is the part that usually trips up most people. They know they want to implement this in their organization. They know how or they can refer to explanations that will suggest how. The problem comes when we try to actually implement the process.

    As is usual with almost any new initiative, we want to follow three precepts:

    1. Start small.
    2. Pick off the easy items first.
    3. Publish the successes.

    Taking these in order:

    Start small
    Start with something over which we have full control or a limited number of stakeholders. Do not start with a mission critical project with numerous stakeholders; that will not work.

    Pick off the easy items first
    A quick analysis will reveal some small and easy to fix items that can be corrected. Sometimes it can be as small as adding a field to a defect report that avoids an extra step further down in the process or removing a step like having Quality Control test a defect as it arrives from the customer before passing it to development and then testing it on the way back out. The second example might have had use at one point but no longer makes sense. A third and frequently easy place to look for improvements is in items that have been made redundant by technology changes. These can include printing items that are now easily stored online or even still testing installation disks for a hosted system.

    Publish the successes
    Even though this is third, it requires some upfront thought. We need to know the current situation and statistics before we can determine whether improvement has occurred. So even though this is the ‘last’ step it must be considered early.

    Rule 3: Take a step back; look objectively at your process; pick off a few items that can be improved; check the current situation; make the change and measure again.

    This is the first step to implementing Right the First Time or Quality Improvement in your organization. Next week; an anonymous example of How Not to be Right the First Time.

  • Quality Assurance – Benefits of Right the First Time

    In the last couple of weeks, we have been discussing the first two steps of Doing it Right the First Time. This week, we will take brief side trip to the Benefits of Right the First Time. A lot of groups stumble over this point because they cannot articulate this well. They seem to use Descartes (as sometimes translated) “I Think therefore I Am” to state that Quality Assurance is “Good because it is” and leave it at that. That argument, in the Quality Assurance case, is circular. One might argue that Descartes is also circular but that is not something we want to take up here.

    There is effort involved in “Getting it Right” and the question is whether it pays back later. We argue that is does using the following reasons:

    1. Having people actually concentrate on each step individually means that they will fully understand that particular piece.
    2. They can project all the possible consequences of that step.
    3. They can determine all the possible precursors that might impact that step.
    4. They can determine all the possible outcomes from that step.

    There are two major counter-arguments to the above. One we are already addressing here. The other can be addressed using the argument we have already stated.

    The first major counter-argument is the lack of time to do this type of analysis. Few people will deny the necessity of doing the analysis so this is only a question of timing and doing it earlier usually is beneficial in informing people.

    The second major counter-argument is that things will change before we are done so why spend the time on something that we will need to redo later. This argument misses the point of the process. The analysis carried out in this process identifies that possibility for change, anticipates what is going to happen, addresses it and moves to the solution.

    Step 3: Make sure we understand the benefits of this process so we can explain it at any time.

  • Quality Assurance – Right the First Time (Part Two)

    Last week we talked about getting it Right the First Time and started with Step 1. The next question many people ask is “How”. Usually they agree that this would be be a good thing for their project or organization and wish it could be done but stumble over How. There is a requirement for a cultural change if this is to occur throughout the organization but it is always possible to start small and implement this within a single project.

    The key points are as follows:

    1. Take the diagram created as part of Step 1 last week.
    2. Concentrate on each piece in turn.
    3. Determine the needs of that piece of the diagram in terms of who is involved; what their involvement is and what they are expected to complete by the end
    4. Look at the process or processes ahead of the one on which we just concentrated and determine what outputs are needed or can be anticipated.
    5. Revise (if necessary) our understanding of the current piece of the diagram based on what can be expected from the previous steps.
    6. Look at the pieces of the diagram that are recipients of the products of the current step and determine their needs.

    With all the above understanding in place, we are now in a position to get each step as close to perfect as we can. Get the relevant stakeholders together and with their undivided attention start deconstructing each step Combining the steps, while still considering any differences or modifications will give us something that is a lot closer to ‘Right the First Time’ than any process that sketches a diagram and starts development immediately based on incomplete knowledge.

    Step 2: Break the high level diagram into pieces and determine who is involved with each step; their involvement; the time and effort required and then get their time and commitment. Now we can launch the next step.

  • Quality Assurance – Right the First Time (Part One)

    Quality Assurance is frequently confused with Quality Control (Software Testing) and many people will sell Quality Assurance Solutions and provide Software Testing. Quality Assurance refers to methodologies and processes that try to avoid errors and mistakes in the first place and getting it right the first time.

    It is also confused (as we have mentioned in the past) with slowing down the process and stopping projects from completing. If this occurs then the Quality Assurance is being implemented at the wrong level. We need to take a step back and look at the overall process to determine where the impediments to completion are occurring and fix those problems. Quality Assurance (properly applied) gets projects done faster and better.

    So what does, ‘Right the First Time’ mean in practical applications.

    The following is a partial list:

    1. It means determining with some degree of accuracy what we are trying to do in this project.
    2. It means recording, in a format that can be distributed, some details of how that solution is going to look on completion. These are frequently referred to as requirements.
    3. It means creating a solution that properly reflects what we want at the end and ensuing that any deviations are understood and accepted.
    4. It means going back at each stage and making sure we have it right before we move forward.

    What does it not mean? (Also a partial list)

    1. It does not mean bringing everyone in to review everything all the time so that everyone gets inured to any errors.
    2. It does not mean mean putting in Checkpoints that bring the project to a complete halt. Checkpoints, properly implemented, are fine.
    3. It does not mean letting everyone do what they want and then piecing it together afterwards and hoping it works

    Your Quality Assurance processes can mean the difference between success and failure; profitability and loss; company survival and the end of the organization.

    Step 1: Make a small diagram of the process you follow in your projects. It should not fill more than one page and is only a high level hand drawn sketch. That provides a high level framework. Next week Step 2.

  • Acting on the Results – Redoing

    Last week we discussed how to measure the results of a Process Improvement initiative and how to successfully implement the process. This week we have assumed that the Process Improvement Initiative has been completed and we are now faced with Acting on the Results.

    There are two possible outcomes from the results and both lead to further action.

    1. The measurements and metrics indicate a successful process improvement and we have achieved the results we want.
    2. The measurements and metrics indicate an unsuccessful process improvement and we have to decide what to do next.

    In the beginning it is usually suggested that we review the measurements and derived metrics to ensure that we are measuring what we thought we were measuring. A false positive, (we think our program was successful but it was not) can be caused by either measuring the wrong item, deriving the wrong metrics or drawing the wrong conclusion. If the analysis indicates that the conclusion is correct then the following steps are recommended:

        1. Determine a new process improvement initiative.
        2. Determine if our existing metrics will properly measure the new initiative.
        3. Define new metrics as required.
        4. Implement and measure as before.

    If, after due analysis, we determine that our process improvement initiative did not lead to the correct result, then we have to decide between the following two possibilities:

        1. Undo the changes made and revert to the previous method of doing the process.
        2. Initiate another process improvement session to change the process to something better.

    For either of the above, we still need to measure and make sure we are accomplishing what is desired in terms of Process Improvement or maintenance of the status prior to the initial change.

    Note that none of this is based on supposition or guessing. No changes are made without supporting statistics.

    Next week: Quality Assurance – Doing it right the first time.

  • Acting on the Results

    This is the first of two blogs devoted to implementing metrics and discusses the piece most likely to cause failure. Assuming that the metrics were properly identified, measured and the results analyzed, we are now at the point of Acting on the Results of the metrics.

    There are two major ways in which a organizations tend to act on results.

    1. Making major changes all at once and cleaning up the results.
    2. Incremental changes

    Making major changes all at once

    In this approach the organization identifies where it wants to be, determines the changes and makes them all at once. This is sometimes called the Big Bang approach, often followed by the Gib Gnab which is a Big Bang in reverse
    There are several steps:

    1. Determine where the end point of the changes would be
    2. Make the changes to get there
    3. Look for what has gone wrong or is incomplete or has been resisted (this will likely be a long list)
    4. Fix the problems

    Incremental changes
    In this approach the organization identifies where it wants to be, determines the changes and decides on individual small steps that will end up getting to the correct point.
    There are several steps:

    1. Determine where the end point of the changes would be
    2. Determine individual small steps that will ultimately lead to the end result
    3. Coordinate the individual steps for each objective
    4. Obtain buy in from the affected parties for the individual steps
    5. Start making the steps
    6. Measure the progress continuously making changes if any measurements show adverse results
    7. Watch for progress towards the final goal

    If the above process is carried out correctly, not only will we get to the final point, we will also have support for our next measurement program. If any of these look unfamiliar give us a call and we will talk about it.

    Next week: Acting on the results – Redoing