NVP Software Solutions

Tag: Software Testing Strategy

  • A Better Way – Case Study 4 – Caught Between Vendor and Client!

    A Better Way – Case Study 3 – Test Plan in a Hurry!

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The fourth case study might be called “Caught between Vendor and Client”.

    Although one could argue that Quality Assurance has always been caught between a Vendor or Vendors (developers) and possibly multiple clients (users), it has become a little more obvious and formal with the purchase of software from outside groups. This is not something that is going to go away. Assembly of a solution rather than building it has been around for quite a while and will probably become more frequent rather than less. The question is what Quality Assurance does to “Bridge the Gap” between what the client wants (‘perfection!’) and what the vendors are willing to supply in terms of proof given competitive secrets and possibly some Non-disclosure requirements.

    In our case (as discussed here) we built a plan that provided the final client with what they wanted. We then mapped what was supplied by the vendors against what was required and filled in the rest. Not surprisingly, the main items that were missing were Integration testing between what the various suppliers had provided and the type of testing that needed the entire system to be there including Performance, Security and Usability (to name a few).

    Bridging the gap in this fashion satisfied everyone and made use of everything that was already in place. That saved us a lot of time and allowed us to concentrate the tests that were critical to the client.

    If you want to discuss this further contact us.

  • A Better Way – Case Study 3 – Test Plan in a Hurry!

    A Better Way – Case Study 3 – Test Plan in a Hurry!

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The third case study might be called “Test Plan in a Hurry!”.

    This issue came up from an organisation that was part way through (as usual!) an engagement and suddenly required a test plan to satisfy the client. The request came from the Project Manager on Friday with a deadline of Monday afternoon. There was no prior exposure to the project; no knowledge that the request was coming; and very little in the way of Project Documentation (with certainly no time to review it). There was a temptation to ignore the request and had they not been an existing client, we might have been tempted to point out that this was not really an effective use of time. However, given the nature of the request and the people from whom it came, we went ahead with the attempt.

    Clearly, we were not going to get detailed testcases or test objectives based on what we had been given. So we opted for a process based Master Test Plan. In other words, we put together a statement of how the project would be tackled from a Quality Assurance point of view when the information became available. We put in processes for all the Quality Assurance items and highlighted the risks inherent in testing under these conditions (including the lack of any understanding of the project) and went forward with that. We put a strong emphasis on what was required now in order to make this work.

    If you want to discuss this further contact us.

  • October QA Events – Toronto and GTA

    NVP Software Solutions will be participating in the following software testing and quality assurance events happening this October in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there! (more…)

  • A better way to Test – 4

    A better way to Test – 4

    In our last blog we discussed addressing Risk via Testing and one of the problems. Today we want to continue with another of the problems. “More is NOT necessarily Better“. Traceability has come a long way in the last few years. Many of the better test management tools automatically include Traceability modules and generate metrics as a result. The fact that they can measure coverage and provide instant statistics has taken a lot of boring work out of managing a testing effort. Where they can lead astray is in assuming that everything has been checked and that the percentages mean something. We recently visited an Assessment site where the raw figure of 83% of the testcases had been completed successfully in the last cycle. The site was planning on raising that percentage to about 93. Considering where they had started a few years ago; it was a laudable aim and a vast improvement. However, there were several pieces missing from the statistics they were using:

    1. Whether any of the testcases were redundant and simply repetitions of each other.
    2. Whether any of the testcases were simply ‘variations on the same thing’. i.e. they were just doing the same thing with slight variations in data
    3. How long since the last review of the testcases.
    4. Whether the testcases actually addressed any of the current risks.

    Testing Faster or doing More testing may not translate into a reduction in risk. It may actually increase the risk by making people assume that they have done a lot of work and a lot of coverage while completely missing major risks.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 2

    A better way to Test – 2

    The second item to determine is the Risk. We always guess that, if you gather three people from the project together and ask them what the risk is, you will end up with 4 answers: Three from the people at the meeting and the right one that no-one mentions. Levity aside, we do need to figure out the real Risk and it is not always obvious.
    Some questions you can ask (and elicit the three answers as a result):

    1. What is the worst thing that could happen?
    2. Who can be hurt the most if something goes wrong?
    3. What is going to cost the most if it goes wrong?

    Regardless of the answer or answers to the above questions, Quality Control is aimed at Risk Reduction. Everything we do needs to be aimed at reducing the risk in the project. Doing that can mean a lot of things that are far beyond traditional testing. We might want to ask what could be done now (before the end of the project) to reduce the risk. We might want to go outside the standard boundaries of the SDLC to consider solutions. This comes back to the need for early involvement. When the project is too far along, it is difficult to suggest radical solutions that might be beneficial. People are too set in their ways and the project is careening to an end that no one likes.

    This is why people who advocate a standard methodology or approach to testing for every project are locked into something that may not work. The same solution does not always work twice!

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 1

    A better way to Test – 1

    The first thing to determine is the type of project, type of methodology being used to build it and the people involved (All of them) in building the project. This is not new by any means and seems obvious information to gather, but in the rush to get at testing when you have been recruited late in the project, this is often either skipped or compressed (usually with dire results later on). A few leading questions or a template that needs to be filled in for the Project Test Plan can often help in ensuring that you get all the information that is available. Frequently we find that this also helps the other people in the project get a better understanding of what they are dealing with and some of the expectations.

    We started into this process at a recent project and elicited the ‘surprising’ information that the design was wrong. The vendor’s exact words were ‘That is not the way our product works’. The project ran a lot longer than was expected and required a lot more testing since the design was revamped at great cost late in the project and the changes had to ripple throughout the software and hardware.

    The above is simply preliminary information that needs to be gathered before we even consider what we are going to do with it and how it impacts our testing.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • Quality Assurance ROI

    One of the items which comes up in most Quality Assurance discussions is the determination of what Return on Investment Quality Assurance provides. It can be hard to justify and obviously depends on the Costing Model used in your organisation. The factors in the Costing Model may differ but the overall trend is always the same. Early Quality Assurance (see last blog) alway pays.

    Here are some of the places where and how Quality Assurance pays:

    1. An error found at Requirements time will be much cheaper to fix there rather than later in the lifecycle. Estimates range from 10% of the later cost down to far less than 0.1% for even moderately complex systems.
    2. Similarly an error found at Design time has a ratio that is slightly less than Requirements but still substantial.
    3. Inclusion of Quality Assurance at an early stage will lead to root cause analysis of repeated errors saving thousands of dollars in testing and repetition of test cases.
    4. Inclusion of Quality Assurance at an early stage allows the definition and construction of an environment suitable for testing when the code becomes available. Environments may be able to be shared leading to cost savings or, at the very least, they can be designed with all the testing considerations included up front.

    The above are only a few of the places where Quality Assurance can save many thousands of dollars on the project. There are many more which could be defined and calculated.
    Our next blog will look at how Quality Assurance can provide valuable input and how that is made clear to the users.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Quality Assurance is exceedingly Cost-Efficient.

    Contact us for further information.

  • Review Design (by Reviewing)

    The title of this Blog may be considered to be a self-evident truth. How else would you review a design except by reviewing it? There are other ways which we will get to in a couple of weeks. However, our initial look will be by the standard method of reviewing them and what a Quality Assurance person might get out of the review.

    Reviewing a design can be difficult for some people. The following problems may arise:

    • You don’t know anything about the technical solution
    • The design is very technical
    • It is difficult to maintain concentration for an extended period of time

    There are ways around these problems. Some of them are personal and some relate to the methodology used to review the requirements.

    On the personal level:

    • Familiarise yourself with the system for which the design is being created. It will help! You cannot review in a vacuum.
    • Organise the design into sections so you can concentrate on a particular section at once.
    • Ask for a System Boundary Diagram (SBD).

    We emphasized one item above. The System Boundary Diagram can be created at many levels but the common idea is that it shows the system in a pictorial form and identifies the interfaces to external systems. The act of creating an SBD brings many problems to the surface and allows the Scope of testing to be set.

    Once you have completed your personnel review, you may be asked to join a group review bringing in your comments and hearing from everyone else. There are several methods of review depending on Risk including Desk Checking, Walkthroughs, Reviews, and finally Inspections. These vary from informal to very formal. However, they are all aimed at finding the errors in the design sooner rather than later. They also all assume on previous individual work.

    For a different way to review the design, look for our next blog on this topic.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Design review is very cost-effective.

    Contact us for further information.