NVP Software Solutions

Tag: software testing

  • Is your Software Testing Ad-Hoc?

    We wanted to start the new-year off with a topic we hear a lot about from many many people.

    Our Software Testing is Ad-hoc (i.e. created or done for a particular purpose as necessary.).

    • It is never reused.
    • We are always looking at testing each project as if it were a brand new experience.
    • Very little gets carried forward from previous projects and a lot of stuff seems to disappear

    If you have heard this or felt this way, you are not alone. The comment that “We had this somewhere but I cannot remember where or cannot find it right now” gets repeated a lot.

    The question is why does it occur. Some of the answers are below:

    • Project budgets are not built with the intent of supplying tests to later projects.
    • No-one can predict whether the same testcases will be needed in a future project
    • No-one can predict whether the testcases will be valid for a future project (may be outdated).
    • It is not possible to estimate how long it will be before an update is needed and we might re-use the testcases.

    All of the above reasons mitigate against creating and retaining robust testcases suitable for future use. The end result is ad-hoc testcases created for the project and discarded after one or a few uses.

    If you want a process that will solve this problem, come back in 2 weeks when we will provide a methodology that will solve this problem at minimal project cost and with positive ROI over the lifetime of the software.

    In the meantime, if you are in the GTA (Greater Toronto Area) or KW, see our next blog next week about the coming presentations.

    If you cannot wait for the two weeks for an answer look at some of the following information:

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter

  • Review of the Year – Manual Testing

    Review of the Year – Manual Testing

    Manual Testing is something that seems to be on the minds of many Quality Assurance Managers and Test Leads. Usually they want out of the Manual Testing and view Automated Testing (Blog planned for next week) as the saviour of their budget and time constraints.

    However, judging by the vacant positions we get requests to fill, there is still no shortage of Manual Testing positions at least in our area. There are still a lot of requests for Manual Testers with business knowledge preferred and new software and startups still start with manual testing. We get requests for Automated Testing with specific tools usually requested and will discuss this next week.

    The part that seems to be missing from many of the requests and the subsequent position is any discussion of the How; What; Why; When; and If; of the manual testing.

    There seems to be limited thought given to How the testing is to be done apart from some vague request to build testcases and execute them.

    Little consideration is given to What to test and Why beyond the statement: “We need to test the software”.

    When and If are not such an issue: Yesterday and definitely are the one word answers to those questions.

    These answers certainly provide freedom for the tester to do what they want but that may not always align with all the stakeholder’s wishes and may be 180 degrees off in some cases.

    This leads to a poor ROI and a large waste of time and money.

    There will continue to be a market for manual testers for new changes and new applications that are not yet mainstream. We expect automation to take over many of the repetitive tasks (as has always been the case) The only open question at this stage might be what AI will do to the industry. That we cannot predict.

    Want to discuss the effectiveness of your Manual Testing further? Contact us.

  • QA with No time

    QA with No Time

    One of the constant issues that comes up in discussions or classes is the lack of time for Quality Assurance and Quality Control. We seem to be under constant time pressure and with Agile and DevOps it has become worse rather than better.

    While Quality Assurance cannot anticipate everything, the Continuous Improvement aspect can help improve the time considerations. The key is to prioritise and plan for the high priority items to be completed.

    Some people will say (with a fair amount of truth) that even planning for the priorities and only doing those high priority items will still not be possible in the time allotted for Quality Assurance and Quality Control. With development reacting to changing requirements from the users, constant upgrades in technology (both hardware and software) and impacts from other projects, the time scale can shrink radically.

    However, without a definitive list of what must be done and the risk of not completing attached, it is very hard to push back against the time pressure. With the list in hand, it is easier to quantify the risk of not doing something.

    Some people will say that they do not have the time to even create the list. They are under extreme pressure to start testing immediately and provide issues so the developers have something to handle now that they have finally delivered all the changes requested by the user. Our recommendation, in this case, is to simply make a list of the functions or requirements, assign High, Medium or Low to the risk of not testing that function or requirement and send it around for review. This will at least alert people to the challenges faced by Quality Assurance and Quality Control.

    Want to discuss your list further? Contact us.

  • A Better Way – Case Study 4 – Caught Between Vendor and Client!

    A Better Way – Case Study 3 – Test Plan in a Hurry!

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The fourth case study might be called “Caught between Vendor and Client”.

    Although one could argue that Quality Assurance has always been caught between a Vendor or Vendors (developers) and possibly multiple clients (users), it has become a little more obvious and formal with the purchase of software from outside groups. This is not something that is going to go away. Assembly of a solution rather than building it has been around for quite a while and will probably become more frequent rather than less. The question is what Quality Assurance does to “Bridge the Gap” between what the client wants (‘perfection!’) and what the vendors are willing to supply in terms of proof given competitive secrets and possibly some Non-disclosure requirements.

    In our case (as discussed here) we built a plan that provided the final client with what they wanted. We then mapped what was supplied by the vendors against what was required and filled in the rest. Not surprisingly, the main items that were missing were Integration testing between what the various suppliers had provided and the type of testing that needed the entire system to be there including Performance, Security and Usability (to name a few).

    Bridging the gap in this fashion satisfied everyone and made use of everything that was already in place. That saved us a lot of time and allowed us to concentrate the tests that were critical to the client.

    If you want to discuss this further contact us.

  • October QA Events – Toronto and GTA

    NVP Software Solutions will be participating in the following software testing and quality assurance events happening this October in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there! (more…)

  • A Better Way to Test – 7

    A better way to Test – 7

    In our last blog we discussed, testing to risk.

    One of the methods we mentioned to address Risk was testing, even if it was not the only method. However, stating that we should test and actually completing the task are two different items. Testcases have to be written or at least understood. Results need to be gathered (in some fashion) and reports produced. Whether you are working very formally and need everything recorded or can just rely on what you have seen and experienced without proof does not matter. You will have to do some testing and to do testing you need some sort of statement of what you are going to do and accomplish.

    We recommend building test conditions or test objectives in order to determine what is going to be tested. These don’t have to be in detail or have all the components of a standard testcase as long as they are commensurate with the risk of the project. This key point seems to be missed quite frequently. People (in particular those who are under time and budget pressure) want to reduce the level and detail of testing until it does not sufficiently address the risk.

    The above reason is why we suggest connecting the testcase to the risk where possible. That makes it much easier to calculate the risk of doing or not doing a particular testcase during a testing cycle. It also, coincidentally, feeds your regression base immediately and allows proper selection when the time runs out.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A better way to Test – 5

    A better way to Test – 5

    In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.

    One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:

    1. Requirements (this comes out of Traceability).
    2. Function tested (may be a subset of the Requirements or it may be a separate mapping).
    3. Date of testcase creation (just so you know how old it is).
    4. Priority of the testcase.
    5. Some statistics generated from the testcase execution.
    6. The risk it tests – we have rarely seen this explicitly listed.

    The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.

    When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 4

    A better way to Test – 4

    In our last blog we discussed addressing Risk via Testing and one of the problems. Today we want to continue with another of the problems. “More is NOT necessarily Better“. Traceability has come a long way in the last few years. Many of the better test management tools automatically include Traceability modules and generate metrics as a result. The fact that they can measure coverage and provide instant statistics has taken a lot of boring work out of managing a testing effort. Where they can lead astray is in assuming that everything has been checked and that the percentages mean something. We recently visited an Assessment site where the raw figure of 83% of the testcases had been completed successfully in the last cycle. The site was planning on raising that percentage to about 93. Considering where they had started a few years ago; it was a laudable aim and a vast improvement. However, there were several pieces missing from the statistics they were using:

    1. Whether any of the testcases were redundant and simply repetitions of each other.
    2. Whether any of the testcases were simply ‘variations on the same thing’. i.e. they were just doing the same thing with slight variations in data
    3. How long since the last review of the testcases.
    4. Whether the testcases actually addressed any of the current risks.

    Testing Faster or doing More testing may not translate into a reduction in risk. It may actually increase the risk by making people assume that they have done a lot of work and a lot of coverage while completely missing major risks.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.