NVP Software Solutions

Category: Quality Assurance Consulting

  • QA with No time

    QA with No Time

    One of the constant issues that comes up in discussions or classes is the lack of time for Quality Assurance and Quality Control. We seem to be under constant time pressure and with Agile and DevOps it has become worse rather than better.

    While Quality Assurance cannot anticipate everything, the Continuous Improvement aspect can help improve the time considerations. The key is to prioritise and plan for the high priority items to be completed.

    Some people will say (with a fair amount of truth) that even planning for the priorities and only doing those high priority items will still not be possible in the time allotted for Quality Assurance and Quality Control. With development reacting to changing requirements from the users, constant upgrades in technology (both hardware and software) and impacts from other projects, the time scale can shrink radically.

    However, without a definitive list of what must be done and the risk of not completing attached, it is very hard to push back against the time pressure. With the list in hand, it is easier to quantify the risk of not doing something.

    Some people will say that they do not have the time to even create the list. They are under extreme pressure to start testing immediately and provide issues so the developers have something to handle now that they have finally delivered all the changes requested by the user. Our recommendation, in this case, is to simply make a list of the functions or requirements, assign High, Medium or Low to the risk of not testing that function or requirement and send it around for review. This will at least alert people to the challenges faced by Quality Assurance and Quality Control.

    Want to discuss your list further? Contact us.

  • A Better Way – Case Study 5 – Distributed with Poor Testing

    A Better Way – Case Study 5 – Distributed with Poor Testing

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The fifth case study might be called “Distributed with Poor Testing”.

    In this case the developers were geographically remote, the Quality Assurance (such as it was) was local and the clients were geographically remote. This was a somewhat unusual situation since it is frequently the Quality Assurance that is remote and the other two local. However, some of the considerations had to be the same. We needed a way of communicating things without losing anything in transit.

    Two solutions came to mind immediately:

    1. Make sure there are stated expectations regarding the flow of information. This applies to any artifact like a test case, a requirement, a defect, or a design document. We did this via some flow charts we placed in the Master Test Plan although there are certainly other places they could have been recorded.
    2. Acquire some tools that will support this information store. There are many free or very cheap cloud-based tools that support these processes.

    Once we had the framework in place, we set up the appropriate loops and feedback mechanisms to ensure information flow and good quality. As time went on we expanded the groups who had access to the systems ensuring that the information flowed in the correct way and with sufficient security to the concerned parties. The two way flow of information allowed us to eliminate a backlog of defects that had accumulated and start addressing customer concerns in a timely manner.

    The investment was not large when compared to the improvements realised.

    If you want to discuss this further contact us.

  • A Better Way – Case Study 4 – Caught Between Vendor and Client!

    A Better Way – Case Study 3 – Test Plan in a Hurry!

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The fourth case study might be called “Caught between Vendor and Client”.

    Although one could argue that Quality Assurance has always been caught between a Vendor or Vendors (developers) and possibly multiple clients (users), it has become a little more obvious and formal with the purchase of software from outside groups. This is not something that is going to go away. Assembly of a solution rather than building it has been around for quite a while and will probably become more frequent rather than less. The question is what Quality Assurance does to “Bridge the Gap” between what the client wants (‘perfection!’) and what the vendors are willing to supply in terms of proof given competitive secrets and possibly some Non-disclosure requirements.

    In our case (as discussed here) we built a plan that provided the final client with what they wanted. We then mapped what was supplied by the vendors against what was required and filled in the rest. Not surprisingly, the main items that were missing were Integration testing between what the various suppliers had provided and the type of testing that needed the entire system to be there including Performance, Security and Usability (to name a few).

    Bridging the gap in this fashion satisfied everyone and made use of everything that was already in place. That saved us a lot of time and allowed us to concentrate the tests that were critical to the client.

    If you want to discuss this further contact us.

  • A Better Way – Case Study 3 – Test Plan in a Hurry!

    A Better Way – Case Study 3 – Test Plan in a Hurry!

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The third case study might be called “Test Plan in a Hurry!”.

    This issue came up from an organisation that was part way through (as usual!) an engagement and suddenly required a test plan to satisfy the client. The request came from the Project Manager on Friday with a deadline of Monday afternoon. There was no prior exposure to the project; no knowledge that the request was coming; and very little in the way of Project Documentation (with certainly no time to review it). There was a temptation to ignore the request and had they not been an existing client, we might have been tempted to point out that this was not really an effective use of time. However, given the nature of the request and the people from whom it came, we went ahead with the attempt.

    Clearly, we were not going to get detailed testcases or test objectives based on what we had been given. So we opted for a process based Master Test Plan. In other words, we put together a statement of how the project would be tackled from a Quality Assurance point of view when the information became available. We put in processes for all the Quality Assurance items and highlighted the risks inherent in testing under these conditions (including the lack of any understanding of the project) and went forward with that. We put a strong emphasis on what was required now in order to make this work.

    If you want to discuss this further contact us.

  • A Better Way – Case Study 2 – Thinking Like the Client

    A Better Way – Case Study 2 – Thinking Like the Client

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The second case study might be called “Thinking Like the Client”.

    Even though many in IT have embraced Agile in some form or another and with some level of success, not all clients are so willing. We find this particularly the case with larger, older and more safety or image conscious clients. They are not willing to take the risk of everything not coming together at the correct time. They are not willing to live with constant change. In addition, they often have milestones that need to be fulfilled and quite frequently they have stated contract deliverables that need to be fulfilled so that payments can be made.

    We received a call from the President of a development company with a large contract with probably the oldest, largest and most image conscious client you could get – the federal government. Even though the company had delivered projects successfully before and had a good record of coming through with what was required, the client still wanted a formal Test Plan completed.

    In our experience, the client is normally okay with a High Level Master Test Plan provided it contains the following (this is not an exhaustive list by any means):

    1. High Level Objectives
    2. A listing of the types of testing needed and the risk around each type
    3. Processes for Test Creation and signoff
    4. Processes for Defect tracking and resolution
    5. Methodologies for executing the testcases

    This reassures them that the developers have an understanding of what needs to be done.

    If you want to discuss this further contact us.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A Better Way – Case Study 1 – Thinking Outside the Box

    A Better Way – Case Study 1 – Thinking Outside the Box

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The first one revolves around a junior QA, a very successful (small) company and a need to test effectively and quickly for a large final client. The issue that came up was the ability to “Think outside the box”.

    We received a call from the President of the company indicating that the new junior QA was having trouble considering things “Outside the box”. They were good with what was presented in the Use Cases or User Stories. Most people can generate testcases based on what is provided and the happy path from the Use Cases and User Stories. Experienced testers will apply other techniques and may specifically try Exploratory testing. However, you cannot explore or test what you don’t know or don’t think of at the time.

    Rather than dictating “Outside the box” which is a contradiction in itself, we decided to go with more of a checklist approach listing some of the areas that would be considered to be non-Happy Path and see if it could lead to further extensions. We did not dictate everything but started with more of a charter and guidance list to see what would come of it. The process had two advantages:

    1. We would have some coverage of everything “inside” and “outside” the box.
    2. We could use the results to evaluate the suitability of the junior QA for further roles and projects.

    The President came up with the basic list and ran it from there.

    If you want to discuss this further contact us.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A better way to Test – 5

    A better way to Test – 5

    In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.

    One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:

    1. Requirements (this comes out of Traceability).
    2. Function tested (may be a subset of the Requirements or it may be a separate mapping).
    3. Date of testcase creation (just so you know how old it is).
    4. Priority of the testcase.
    5. Some statistics generated from the testcase execution.
    6. The risk it tests – we have rarely seen this explicitly listed.

    The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.

    When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 4

    A better way to Test – 4

    In our last blog we discussed addressing Risk via Testing and one of the problems. Today we want to continue with another of the problems. “More is NOT necessarily Better“. Traceability has come a long way in the last few years. Many of the better test management tools automatically include Traceability modules and generate metrics as a result. The fact that they can measure coverage and provide instant statistics has taken a lot of boring work out of managing a testing effort. Where they can lead astray is in assuming that everything has been checked and that the percentages mean something. We recently visited an Assessment site where the raw figure of 83% of the testcases had been completed successfully in the last cycle. The site was planning on raising that percentage to about 93. Considering where they had started a few years ago; it was a laudable aim and a vast improvement. However, there were several pieces missing from the statistics they were using:

    1. Whether any of the testcases were redundant and simply repetitions of each other.
    2. Whether any of the testcases were simply ‘variations on the same thing’. i.e. they were just doing the same thing with slight variations in data
    3. How long since the last review of the testcases.
    4. Whether the testcases actually addressed any of the current risks.

    Testing Faster or doing More testing may not translate into a reduction in risk. It may actually increase the risk by making people assume that they have done a lot of work and a lot of coverage while completely missing major risks.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.