NVP Software Solutions

Tag: QA

  • A Better Way – Case Study 2 – Thinking Like the Client

    A Better Way – Case Study 2 – Thinking Like the Client

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The second case study might be called “Thinking Like the Client”.

    Even though many in IT have embraced Agile in some form or another and with some level of success, not all clients are so willing. We find this particularly the case with larger, older and more safety or image conscious clients. They are not willing to take the risk of everything not coming together at the correct time. They are not willing to live with constant change. In addition, they often have milestones that need to be fulfilled and quite frequently they have stated contract deliverables that need to be fulfilled so that payments can be made.

    We received a call from the President of a development company with a large contract with probably the oldest, largest and most image conscious client you could get – the federal government. Even though the company had delivered projects successfully before and had a good record of coming through with what was required, the client still wanted a formal Test Plan completed.

    In our experience, the client is normally okay with a High Level Master Test Plan provided it contains the following (this is not an exhaustive list by any means):

    1. High Level Objectives
    2. A listing of the types of testing needed and the risk around each type
    3. Processes for Test Creation and signoff
    4. Processes for Defect tracking and resolution
    5. Methodologies for executing the testcases

    This reassures them that the developers have an understanding of what needs to be done.

    If you want to discuss this further contact us.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A better way to test – 6

    A better way to Test – 6

    In our last blog we discussed, as the last point in our list, testing to risk.

    If you have the risks listed, explained, classified, calculated, ordered and entered into a database, it may be easy to attach testcases to them and determine when you have addressed the risk with testing. Not all risks can be efficiently addressed by testing, it is simply one of the possible techniques to be used.

    However, people rarely agree on the risks and, as mentioned above, not all risks can be addressed by testing. So this needs to be flexible.

    If you don’t have your risk recorded then the testcases may still be built to address perceived risks but they may be difficult to justify. Having both sides of the need (the risk and the testcase) can make each justify the other.

    Once we have this for one release of the software, the only question is whether we can continue using the same testcases from release to release. Risks will change, some will disappear or be downgraded, new ones will appear or the ones we missed last time will be included for future releases. Like all project requirements, nothing is ever perfect and we progress (sometimes unevenly) to an end result of coverage that is sufficient for the project. Of course something will always surprise us.

    The key consideration is not to become inured to the other methods of addressing risk – testing is not the end solution for everything.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 5

    A better way to Test – 5

    In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.

    One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:

    1. Requirements (this comes out of Traceability).
    2. Function tested (may be a subset of the Requirements or it may be a separate mapping).
    3. Date of testcase creation (just so you know how old it is).
    4. Priority of the testcase.
    5. Some statistics generated from the testcase execution.
    6. The risk it tests – we have rarely seen this explicitly listed.

    The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.

    When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 4

    A better way to Test – 4

    In our last blog we discussed addressing Risk via Testing and one of the problems. Today we want to continue with another of the problems. “More is NOT necessarily Better“. Traceability has come a long way in the last few years. Many of the better test management tools automatically include Traceability modules and generate metrics as a result. The fact that they can measure coverage and provide instant statistics has taken a lot of boring work out of managing a testing effort. Where they can lead astray is in assuming that everything has been checked and that the percentages mean something. We recently visited an Assessment site where the raw figure of 83% of the testcases had been completed successfully in the last cycle. The site was planning on raising that percentage to about 93. Considering where they had started a few years ago; it was a laudable aim and a vast improvement. However, there were several pieces missing from the statistics they were using:

    1. Whether any of the testcases were redundant and simply repetitions of each other.
    2. Whether any of the testcases were simply ‘variations on the same thing’. i.e. they were just doing the same thing with slight variations in data
    3. How long since the last review of the testcases.
    4. Whether the testcases actually addressed any of the current risks.

    Testing Faster or doing More testing may not translate into a reduction in risk. It may actually increase the risk by making people assume that they have done a lot of work and a lot of coverage while completely missing major risks.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • Selling QA

    In the last couple of blogs we talked about Early Involvement and the ROI of Quality Assurance. A list of a few places where Quality Assurance pays was also provided.

    However, Selling Quality Assurance can still be a challenge.

      • Some organisations see the benefit and buy in to the concept permanently.
      • Others never make the move towards a Quality Culture and cannot see the need.
      • The most common scenario is a cyclical one:

    Someone in an organisation embraces Quality and either are in the correct position to make it happen or have great timing where there has been a move towards Quality Assurance and it just needed someone
    to ‘ride the wave’ to completion.

    • There is, of couse, always the last scenario where there has been a crisis and everyone comes to the conclusion “If we only had had Quality Assurance the ‘problem/crisis/disaster’ would never have occurred”.

    Be very careful of the last one; it is not likely a deep commitment. Once the crisis is over, there will be a reversion to the earlier behaviour. It is very tempting to go to an organisation that has just had a crisis and try to convince them to change but it is not likely to occur and will be a struggle.

    So, how do we sell QA:

    1. We need to know the state of the organisation. Is there an underlying need or wish; or just a superficial desire?
    2. Who are the decision makers and what do they want?
    3. Where can be we get a reasonable return in a short period – we have to show something fairly quickly.

    Remember a culture change takes years. We need a set of successes that are directly attributable to Quality Assurance to start making the case for the wholesale introduction and embedding of the concept.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Quality Assurance is exceedingly Cost-Efficient.

    Contact us for further information.

  • Quality Assurance ROI

    One of the items which comes up in most Quality Assurance discussions is the determination of what Return on Investment Quality Assurance provides. It can be hard to justify and obviously depends on the Costing Model used in your organisation. The factors in the Costing Model may differ but the overall trend is always the same. Early Quality Assurance (see last blog) alway pays.

    Here are some of the places where and how Quality Assurance pays:

    1. An error found at Requirements time will be much cheaper to fix there rather than later in the lifecycle. Estimates range from 10% of the later cost down to far less than 0.1% for even moderately complex systems.
    2. Similarly an error found at Design time has a ratio that is slightly less than Requirements but still substantial.
    3. Inclusion of Quality Assurance at an early stage will lead to root cause analysis of repeated errors saving thousands of dollars in testing and repetition of test cases.
    4. Inclusion of Quality Assurance at an early stage allows the definition and construction of an environment suitable for testing when the code becomes available. Environments may be able to be shared leading to cost savings or, at the very least, they can be designed with all the testing considerations included up front.

    The above are only a few of the places where Quality Assurance can save many thousands of dollars on the project. There are many more which could be defined and calculated.
    Our next blog will look at how Quality Assurance can provide valuable input and how that is made clear to the users.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Quality Assurance is exceedingly Cost-Efficient.

    Contact us for further information.

  • Early Involvement of Quality Assurance

    One of the items which comes up in most Quality Assurance discussions is the determination of when to involve Quality Assurance in the project. Those with the traditional view of Quality Assurance as simply Software Testing (Quality Control) delay the arrival of the Qaulity Assurance Professional until the last possible moment and are then disappointed when the testing is not done correctly or with due regard for risk. People (including some Quality Assurance personnel) believe there is nothing to be done until the software is ready and ‘complete’.

    These people are missing the value that can be contributed by Quality Assurance early in the project including the following:

    1. Assessment of Risk from a Quality Assurance point of view.
    2. Review of Requirements and Design from a Quality Assurance point of view.
    3. Proper design of the test environment and the processes surrounding it.
    4. Proper and complete design of the testcases that address the most important risks while minimising the efforts.
    5. The chance to have feedback from other project members over the course of the test design and preparation.

    Of the above items the last is probably the most important. Feedback can help find and correct deficiencies earlier rather than later.

    Our next two blogs will provide concrete examples of how and where Quality Assurance can provide valuable input.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Design review is very cost-effective.

    Contact us for further information.

  • Review Design (by Writing Testcases)

    One of the easier methods of review, for a Software Tester, is to write test conditions, test objectives, or complete testcases for the design. We define Test Conditions or Test Objectives as being statements of what is to be tested without the details usually found in testcases.

    For example the following might be a few High Level Test Objectives for a design:

    1. Test that the system continues to function in the event that the link to the Postal Code database is not working.
    2. Test that the system switches to the backup node when the primary node is taken offline.
    3. Test that the database is updated with the full CDR at the end of the call.

    There is a lot of detail under these Test Objectives and potentially they could generate many, many testcases but they are all real examples and they are all necessary. They key is to look for the following while you are writing the test objectives:

    1. Two, or more, test conditions that contradict each other.
    2. Places where we find it difficult to write any objectives at all – this may be related to a lack of knowledge or the design may be incomplete. Either are issues.
    3. Places where we are guessing at the expected behaviour for our Objective. We do not expect details here but a feeling that we don’t know what the result will be or how to prove it.

    Brainstorming with other people, talking to the System Architect, looking at other similar designs and looking up technical information are all good sources for building these objectives.

    Once you have completed your personnel review, you may be asked to join a group review bringing in your comments and hearing from everyone else. There are several methods of review depending on Risk including Desk Checking, Walkthroughs, Reviews, and finally Inspections. These vary from informal to very formal. However, they are all aimed at finding the errors in the design sooner rather than later. They also all assume on previous individual work.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Design review is very cost-effective.

    Contact us for further information.