NVP Software Solutions

Category: Software Testing

  • Does your software Testing Pay – Part 4

    “Does your Software Testing Pay” is the question we posted six weeks ago. We posted a simple calculation for a ROI and then a few comments that were received as feedback. Those comments were appreciated since they extended the idea past the direct defect costing calculation. Today we want to wrap up this series with some general comments that extend the idea a little farther.

    For the most part we have concentrated on the Software Testing aspect of Quality Assurance. Almost everyone we speak with is familiar with this aspect. Most software companies or those who depend on software complete some form of testing and the concept of testing can be generalised to cover a multitude of activities. However, it is not the entire picture.

    So, at the risk of making the title slightly inaccurate, what other aspects of Quality Assurance can be considered:

    1. Process improvement for the entire SDLC.
    2. Timely intervention at the Root Cause of many of the defects to prevent them from occuring in the first place.
    3. Concentration on the required end-result to make sure everyone is working towards that end. It is suprising how often this is obscured in large organisations. We attended a course a couple of weeks ago and spoke to the instructor afterwards and apparently many people on the course have only been told a very small piece of what they need to do (and part of that was to attend the course although the reason why was not provided). Not only is that against the concepts of keeping people informed of why they are doing something but it is also very de-motivating.

    The above list (particularly number 1) can cover a lot items that are very detailed and can add a lot of value to the Quality Assurance efforts. We may take this up in the fall.

  • Does your software Testing Pay – Part 3

    “Does your Software Testing Pay” is the question we posted four weeks ago. We then posted a simple calculation for a ROI two weeks ago.
    After we posted the second blog we had a couple of comments to the effect that concentrating on defects was a very narrow focus in terms of cost recovery and benefits. This was certainly a valid criticism; we had concentrated on something that could be answered and calculated without too much effort.

    Some of the suggestions that came back were as follows:

    1. Contributing to a better product using the information gleaned from testing
    2. Enhanced knowledge of the product for both testers and developers.
    3. Future design improvements.
    4. A better quality product.

    No doubt more items could be added to the above list

    Now we just need to cost them

    Since some of these are subjective benefits, it is suggested that they be documented and then all stakeholders can assign them to a value bucket (independently). As a starting point we can use an averaged value for each benefit and then convert that into a dollar value to determine the benefit.

  • Does your software Testing Pay – Part 2

    “Does your Software Testing Pay” is the question we posted two weeks ago.
    There are obviously two parts to this question, one is how much it costs and the other is how much it saves.

    The cost portion is reasonably easy:

    1. Chargeback rate on resources (means hours per release must be recorded).
    2. Equipment and space usage (if not included in the above).
    3. Test Tool cost (amortized over all the projects)
    4. Opportunity cost if the resources should be engaged in something else.

    The savings portion is somewhat harder:

    What would each found defect have cost to fix in production? This is obviously an estimate.  One calculation is supplied below.

    1. Look at each defect found by testing.
    2. Estimate the probability of it occuring after release of the code to the users. You might want to take estimates from several people and average them. Either the probabilities must be between 0 and 1 or else you need to convert to a figure between 0 and 1 before using it.
    3. Estimate the cost to the organisation under the assumption that the defect does occur in production. This cost includes the direct costs to the company (fixing, testing and deploying); the indirect costs (administration etc) that are often hidden (Iceberg – 9/10 of the costs are hidden)
    4. The cost of the customers in rework, lost data, inability to respond proerly.
    5. Add up the costs per defect and multiply by the percentage.
    6. Add up the resulting figure for all defects found by testing for a release.

    If Savings > Costs your software testing is paying for itself.

  • March 2019 QA Events in the GTA and Beyond

    If you are in the Greater Toronto Area or Kitchener-Waterloo you might want to consider these events to network with other QA people or learn some of the new ideas in QA.


    NVP Software Solutions will be participating in the following software testing and quality assurance events happening this March or April in Ontario, Canada. The events are located in Toronto and Kitchener-Waterloo in the coming weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there!

    This image has an empty alt attribute; its file name is TASSQ-1.png



    WHY AND HOW TO ADOPT BDD IN SOFTWARE DEVELOPMENT – Rob Ashall- 26 March, 2019
    See https://www.toronto-assq.com/ to register
    This image has an empty alt attribute; its file name is cropped-kwsqa_withicon_small.png


    KWSQA – Kitchener Waterloo Software Quality Association – Wednesday, April 3, 2019 – Making the Web a Little More Accessible – Samantha Campbell  – See https//www.kwsqa.org/kwalitytalks  to register

    Courses

    Plan your 2019 Courses today. See nvp.ca/services/training for a complete schedule.

  • Does your Software Testing Pay?

    “Does your Software Testing Pay” is another question (in the series we are addressing here) that comes up quite frequently. Although it may not be stated directly it can manifest itself in the following actions:
    1. Lack of resources for testing.
    2. Lack of time provided for testing.
    3. Lack of suitable hardware on which to conduct testing
    4. Lack of software testing tools.
    5. …
    This is not a comprehensive list but it covers many of the major manifestations of not having an answer as to whether the testing pays. If the question cannot be answered, then it is difficult to justify expenses or investment in testing. In addition, some Quality Assurance people go to great lengths to avoid answering this question.
    Come back in two weeks to see one way of addressing this question. It will require research, metrics, and statistics.

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter

  • Are you satisfied with your testing – Part 2

    You may recall from the blog of two weeks ago that may organisations end up dissatisfied with their testing.

    The key to resolving this, from Quality Assurance, is to plan your testing before your start. Decide on what must be done, what should be done and what need not be done before the project gets very far.

    Sometimes people call these decisions ‘tradeoffs‘ since they imply that something is being traded off against something else and someone is losing out. Tradeoffs are different and do have the characteristics mentioned in the previous sentence. Here we are planning for what needs to be done.

    Other people claim they need to see the software in order to know what to test. At the detail level this can be true but it is not true at the upper levels.

    Still others will claim that they will think of all the testing that needs to occur while they are doing it. This is not a bad method as long as the tester fully understands all the business, technical, and software requirements and can handle all of this. Small projects with little risk can be done this way. Larger projects with higher risks are not so easy.

    A Quality Assurance process considers all the relevant items at the start and does not wait for a crisis to occur or for management to worry about what has been completed or not completed. It is determined at the beginning and the decisions taken at that point, not in the last 10% of the project with a huge amount of the work to complete. Ongoing reporting and process improvement ensures this works properly.

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter

  • Are you satisfied with your testing?

    Following on from the theme of last month, many organisations are dissatisfied with their testing. They feel it is incomplete, or ineffective or costs too much. At the end of the testing effort they are left with a vague feeling of unease. Often it is difficult for people to quantify their concerns but they are very real and lead to delays and ongoing expensive testing in an effort to remove this feeling.

    The trouble is that the more they do in testing, the more they may realise what they have not done. This does not increase the confidence level! Furthermore, if they do find problems during this ‘extra’ testing effort the level of confidence drops commensurately and even more testing is required until there is no energy, budget or time left.

    Come back on February 25 to see how a Quality Assurance process can address these concerns long before they become issues.

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter

  • Is your Software Testing Ad Hoc – Part 2?

    We wanted to start the new-year off with a topic we hear a lot about from many, many people.

    A couple of weeks ago we talked about the issue of Software Testing being Ad Hoc and some of the reasons this might be the case. See the blog from January 14, 2019.

    The major concerns were loss of information, repetition and reinvention.

    The quickest way to stop this loss is to classify the testcases. This does not have to be complex at the beginning. You can classify the testcases based on the following criteria:

    1. Priority – High – must be executed with every release; Medium – must be executed at least once a release; Low – may be ignored unless there is a lot of time.
    2. Feature/Module/Function – This does require some thought at the beginning since you want to get the classification right to avoid having to do a lot of reclassification later.
    3. Level of testing – Unit, Integration, System, Acceptance – again something that might require some thought at the beginning to avoid having to reclassify things later.

    Obviously this is best done in a tool with a database behind it although a lot of companies start with a spreadsheet. The only expectation is that the next iteration of the project can at least see what exists and pick up what they need.

    Later you may determine other classifications that are useful. If you do, you may or may not want to reclassify everything that has already been stored. You may also want to do it over time as the testcases are used rather than trying to do it as one project.

    We have gone one step towards eliminating the ad hoc nature of the testing and now have something that can be reused.

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter