NVP Software Solutions

Tag: Automated testing

  • Quality Assurance 2019 Review

    Quality Assurance is, as usual, up and down. Some organisations are buying in to Process Improvement, looking long term, and many are realizing that the solutions are not necessarily more testing or another test tool. They understand that the problems lie in how things are done rather than the tools used.

    However, many organizations are still adding test tools or adding test phases and layers without adding value to the Software or the Product.

    We completed several assessments over the past year and provided seminars aimed at process improvement. We are now up to the stage of implementing the process improvements identified. These organizations are moving ahead and planning for the long term. They can see the value of improvement.

    However, on the other hand, some of the people we are dealing with are struggling to complete the necessary testing without considering the long term improvement. They are challenged by the necessity of getting testing done for clients and do not think they have the time for process improvement.

    So will this change in the 2020? It seems unlikely in view of the above unless we consider the following:

    1. Calculate the real benefit of implementation of process improvement over multiple projects. This is the long term, high level, view of how process improvement benefits multiple projects.
    2. Define and gather the appropriate measurements and metrics to determine whether the process improvement is having the desired effect and also to identify further process improvement opportunities.
    3. Select and acquire the appropriate technology to support the process improvement effort. Talk to us about a well tested methodology for QA technology acquisition.

    Photo by Bekir Dönmez on Unsplash

  • Automated Testing

    Judging from what we have heard, AI seems to be the favourite flavour for Automated Testing in 2019. Many of the tools are stating that they are AI enabled or AI enhanced and claiming massive productivity gains as a result.

    We are still seeing tools that require a lot of technical knowledge and other ones that have hidden the actual scripting from the users.

    One other major change is that most tools are now automatically offering either a cloud based or an on-premise solution to suit every client’s wishes and, more importantly, their security needs.

    Tools go through a cycle every few years. For a while we get a different test tool for every possible situation, then someone comes and consolidates a large number of tools into one tool that addresses most situations. Then the cycle starts again. Obviously this cycle is driven by the technology used to build applications and what level of testing is needed.

    In addition to the technology that is used to build any particular product, the methodology also impacts the way automation is applied. Any iterative methodology has vastly different needs from a standard Waterfall methodology.

    What we are not seeing, is much initial analysis to select the best test tool for an organisation. We are still seeing selection based on features and not necessarily on functionality that is applicable throughout the organisation. We are seeing some ROI for individual automation efforts but that is after the tool is selected and implemented. However, as discussed last week, that calculation is not over the entire organisation or over an extended time period. This means we are losing out on some automation opportunities and completing some based on a false calculation.

    So will this change in the 2020? It seems unlikely in view of the above unless we consider the following:

    1. Calculate the real benefit of implementation of automation over multiple projects and years.
    2. Find an automation tool that suits your situation. There are many good ones around; you just need to find the appropriate one. Talk to us about a well tested methodology for test tool acquisition.

    Photo by Franck V. on Unsplash

  • Manual Testing

    This year is the n’th time we have heard about the demise of Manual testing and the n+1’st time it has not occurred.To paraphrase Mark Twain “Rumours of the death of Manual Testing have been greatly exaggerated”.

    Why does this keep coming up year after year:

    1. We keep inventing new items that are not amenable to being automated.
    2. New startups have neither the time or the budget to worry about automating testing. Their emphasis is on getting the product out the door and into the hands of their customers.
    3. Some organisations have a great deal invested in an old automated tool. They are not maintaining the existing scripts or adding new but no one is willing to throw them out.
    4. Some testing tools have not lived up to their promises and people are unwilling to try again with a new test tool.
    5. Most project managers do not have budget for automation of testing and since they do not benefit from it (the next project benefits) they see little reason to add it to their project.
    6. If it becomes a corporate or central responsibility to automate, then the question of funding it becomes awkward. Who is responsible for the cost fo the tool and the automation effort? How is that cost amortized and apportioned?
    7. It appears cheaper to get Manual Testers.

    So will this change in the 2020? It seems unlikely in view of the above unless we consider the following:

    1. Calculate the real cost of repeatedly executing the same testcases manually.
    2. Calculate the real benefit of implementation of automation over multiple projects and years.
    3. See whether the automation will pay for itself using the above two figures.
    4. Find an automation tool that suits your situation. There are many good ones around; you just need to find the appropriate one. Talk to us about a well tested methodology for test tool acquisition.

    Photo by Hunter Haley on Unsplash

  • Look ahead to 2019

    So what is coming in 2019. No doubt, if that could be predicted accurately, we would not be doing this right now! We would be somewhere on a warm sunny beach.

    The easy answers are:

    1. Manual testing will continue.
    2. More automation will be done and much more will be wanted (with or without the understanding of what it involves).
    3. The software we are testing will evolve.

    The harder predictions are the external events:

    1. Increasing legal and regulatory requirements for testing can be expected. While some specific regulations in particular industries will be rolled back , we expect the general direction to be more external requirements relating to legal liability and fulfillment of specific requirements.
    2. New industries will move mainstream with software and subsequent need for testing. (If we knew which ones, we would tell you.)
    3. There will be no letup in the pace of software testing. Items that are ‘old’ and mainstream may be assumed to be okay and require minimal testing but new items will still require testing.
    4. The level of interaction between programs will continue to increase.
    5. AI will impact testing.

    Things we would like to see (our wishlist):

    1. Increased emphasis on ensuring the customer’s needs are met (after all they are paying the bill).
    2. Better understanding of what Quality Assurance and Quality Control has to offer. It is still too late and an afterthought and those that go into the testing with the assumption that they know it all will not cover what they need to cover to reduce the risk.

    One thing that might come up is the necessity (because of volume) to make some code pieces ‘bulletproof’. No matter what you throw at them, they will operate in the way that is expected – either via processing or rejecting the provided information.

    Want to discuss 2019?

    Contact us or join the discussion.
    Past Blogs
    Monthly Newsletter

    Happy New Year.

  • Review of the Year – Automation

    Review of the Year – Automation
    Automation of software testing is something that seems to be on the minds of many Quality Assurance Managers and Test Leads. It has been a popular topic for many years.

    Currently we get requests for particular tools and knowledge of their attributes in particular environments; these are usually serviceable. Current status seems to be a separate tool for every need and environment and sometimes every organisation. Based on past experience, in a few years, someone will consolidate all the disparate technologies under one umbrella tool. Then the cycle will start again with people inventing new tools for specific niches and products.

    We also receive requests for people to “automate our testing” with no decision on the tool attached. This is a completely different question and requires some discussion to occur before the attempt to automate even starts. We need to know the what; when and Why the company wants to automate their testing. The thing we want to avoid are the actions below.

    1. Purchase Automated Test Tool.
    2. Install Tool.
    3. Wait for successful automation to save all the cost of the tool and cost of manual testing.
    4. Become disillusioned.
    5. Go to 1; Repeat ad infinitum.

    This has occurred over and over in different organisations. A lot of money gets used up with no progress and eventually the organisation gives up on the cycle and continues manual testing (see last week’s blog).

    Our best recommendations are as follows:

    Look up where you are on the technology maturity level with your current technology.

    Decide what you need (criteria are available) in terms of automation and what you are capable of handling based on maturity level.

    Then do a Plan to implement your automation. Never assume it will just occur. It won’t.

    Want to discuss how to automate effectively? Contact us.

  • March Events in Software Testing in the GTA and beyond

     

    NVP Software Solutions will be participating in the following software testing and quality assurance events happening this March in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there! (more…)

  • So you want to be a software test automator?

    So you want to be a software test automator and don’t know how to start.

    There are plenty of ways to become a software test automator and we’ve talked to a number of people curious about how to break into the field. If you ask current test automators what their background is here are some answers you may get: (more…)

  • Malthus on Testing – Rapid Testing

    Malthus on testing is not something that you hear often. Thomas Malthus was an English cleric and scholar, who said “The power of population is indefinitely greater than the power in the earth to produce subsistence for man”.

    Last week we discussed how Automated Testing has met or is going to meet its Malthusian Catastrophe fairly soon. While there is no doubt that automation has added greatly to the volume of testing that can be completed, it can be argued that the exponential increases in functionality that are easily programmed but not see easily (fully) tested will eventually overwhelm the scripts.

    Some people will advocate other methods of testing under one heading or another without explicitly mentioning the fact that everything is Risk Based. If the time or resources or both are limited, then people will use their knowledge to get as much done as possible within the available time and with the available resources. They will assess the risk and make an informed decision about what can be completed in the available time. They will assess the context in which they are working and determine what needs to be done. They will use their knowledge and experience to determine what to do. We have done this for years and there is nothing new about it. There are various techniques that can be followed to make sure that risk is reasonable. There are work processes and methodologies that will help.

    We can apply the above techniques to both are manual and automated testing and buy ourselves some more time. However, it is only a step along the way; not a solution

    Final Series Week: Malthus on Testing – New Paradigm. We will propose a new method or paradigm for testing that will address some of the problems.

    Next series: Communication in Testing,