NVP Software Solutions

Tag: Quality Control

  • Test Training

    Training seems like an obvious topic and not one to which a blog or two could be usefully devoted. However we get a surprising number of questions about training and plan to address a few of them here. The first one is what type of training is offered. We define three broad categories here:

    1. Training related to testing.
    2. Training related to a particular Test Tool.
    3. Application related Training.

    You only have to read the job advertisements to see the expectations related to open positions. You may see a long list of test tools with which the applicant is to be proficient. You will most likely see some reference to a Test Methodology or SDLC. Most job advertisements finish off with some soft skills.

    So how do our three categories relate to day-to-day work?

    Taking them in reverse order:

    Application related Training

    Clearly the more the person knows about the application for which the system was built, the easier it is to understand the risks, define the scope of testing and explain the results to the business. It is also easier to understand the business requirements and expectations.

    Training related to a particular Test Tool

    This type of training is usually supplied by a vendor and can range from an overview of the test tool allowing one to to use it without in-depth knowledge all the way to becoming a technical expert. The only comment is that every tool has been superceded by something else eventually so every tool or technical process will eventually become redundant.

    Training related to testing

    This type of training covers the rest of the requirements. It teaches about SDLC, Communication, Risk, Planning, and Testing to name only a few items.

    Discussion Questions

    1. Do you participate in Training for Testing?
    2. Was it beneficial to the project?
    3. What would you have done differently based on what you know now?

    Next Week: Sources of Information

  • How interactive prototyping can improve QA in the SDLC

    It’s often said that quality must be built in, not added on. But when it comes to the Software Development Lifecycle (SDLC), the reverse often happens: defects are identified late on in the Testing Phase, after coding is done. This means bugs are expensive to fix and solutions are found last-minute, putting quality at risk. Early Lifecycle QA, from requirements definition onward, results in a better software development experience and, hopefully, a better end product.

    But even when Early Lifecycle QA does happen, it’s not always plain sailing: business requirements documents are often scanty and don’t provide QA professionals with enough information; other stakeholders may be resistant to QA specialists coming in and “telling them their job” at the review stage; some requirements are untestable thanks to lack of clarity. And of course things change throughout any project, it’s a fact. Flexibility is a must.

    So how can QA professionals ensure that they can get involved and be effective from the outset of the SDLC and throughout it? Step up interactive prototyping. Using an interactive prototyping tool can facilitate early stage QA and avoid common pain points.

    Requirements definition and gathering

    QA specialists sometimes receive little information on which to base tests at this stage, thanks to paltry requirements or incomprehensible Business Requirements Documentation (BRD). Additionally, QAs are often sent the documentation too late, meaning there’s no time to set up adequate tests. By gathering, defining and gathering requirements using a prototyping tool – requirements can be imported or created directly in the prototype, and all invited stakeholders (including QAs) can add or comment upon those requirements in real-time. Once you have the baseline of requirements, a System Testing Plan can be finalized.

    Interactive requirements and iterative process

    Once the BRD and System Requirements Specification are agreed upon, the QA team can set about reviewing requirements in the prototype. Running user test cases with a designated User Proxy – someone who takes on the role of User – will allow QA to be approached from 3 angles: functional, structural and conformance. All QA team members can add to and edit the BRD in the prototype, ensuring that user and system needs are accurately represented at this early stage.

    Using a prototyping tool to facilitate this process reduces time and budget concerns for project managers, which means they are more likely to agree to incorporating QA teams early on.

    Design and QA

    With a version history of requirements accessible within the prototype, the design team has a clear map to work off. They can build an interactive prototype based on the validated requirements, linking each feature to its relevant requirement and thereby facilitating QA testing. Once the design team has produced a high fidelity prototype, activities such as verifying system architecture and carrying out system audits can be done on the prototype. Finding and fixing bugs through prototype testing is a lot cheaper than fixing them in the code.

    Coding and Deployment

    Later SDLC stages can now go ahead, with the QA team carrying out coding-related Quality Assurance activities such as verifying implementation of top requirements, and checking the quality of code with Product Quality Analyzer tools.

    Key Success Markers

    Early Lifecycle Quality Assurance requires collaboration between teams and a shared vision, factors supported by the inclusion of interactive prototyping in the SDLC. By prioritizing Early Lifecycle QA rework and costs are reduced, QA input is incorporated at every stage of the project, and time to market is optimized.

    Justinmind is a prototyping tool for web and mobile applications that allows you to visualize your software solution before starting development

  • Scheduling Test Cycles

    Scheduling Test Cycles often seems to create challenges for Managers, so we thought we’d tackle this for today’s blog. In our experience, there seems to be an ingrained view from Test Managers and Development Managers to not leave time between the Test Cycles or the Fix Cycles for the other party to do their work.

    I have seen Test Cycles scheduled consecutively with no room to actually fix anything. The idea was that they could fix the bug overnight or during the weekend because nothing could impede the test effort at this stage. The alternate problem is scheduling by the Development Manager who puts all the time into the Fix or upgrade time and allocates nothing for testing. The same question elicits a similar answer that testing can proceed overnight or on weekends.

    What is obvious is that there has to be compromise on both sides.

    However, it is possible to schedule overlap. Certainly Developers can start fixing bugs found early in the test cycle. before the cycle is finished, and it’s probably better that they do. However, this requires strong promotion and code control procedures and a plan on how the environments are going to organize. Otherwise, fixes start getting into the test environment before other testing is done. Similarly testing can continue even when Developers are in fix-mode. Planning is required to cover items they aren’t working on at the moment.

    We just need to plan our way through this with the understanding that there will be changes as the project evolves, dependencies arise, and items change.

    Discussion Questions

    1. Have you Scheduled Test Cycles?
    2. If yes, to number 1, how did it work out?
    3. What would you have done differently based on what you know now?

    Next Week: Final meetings for the year

  • Test Run

    Our latest blog will discuss the Test Run. For today’s purpose, NVP considers a Test Run to be one, single execution of a testcase. This could mean that the testcase ran to completion and the expected AND actual results were identical, or that the case the testcase did not have actual results that equalled the expected. We have stayed away from the words ‘successful’ and ‘unsuccessful’ since some may feel a testcase is only successful if it uncovers a problem and is unsuccessful if it does not.

    We are interested in this statistic of test runs for a number of reasons:

    1. It helps in estimation
    2. It helps justify the time taken to test
    3. It provides a measure of code stability

    Estimation

    Knowing the number of Runs of a testcase helps determines how long the cycles and the whole test effort will take next time. If we know we had to run each testcase an average of 5 or 6 times before it ran to completion without raising an issue then we know how many times we may need to run it next time. Note that unsuccessful runs may include attempts that lead to fixing the testcase or relevant test data. Once we have ‘debugged’ the testcase, these runs may not recur.

    Justification

    If we only report the count of completed testcases with actual results equalling expected results, then each testcase might only show a single execution. This would hide a lot of work and effort and make the testers appear very unproductive. Showing that each testcase was executed 6 or 7 times before we were satisfied gives a much better idea of the effort involved.

    Code Stability

    If a testcase is run a dozen times and only on the last time does it run to completion with Expected Results equal to Actual Results, then we may have a concern with code stability or whether that final run was really correct. Something that fails a dozen times and then is successful is highly suspect. Maybe the conditions changed, maybe we missed something, maybe the issue was finally fixed. Whatever the case, we are not sure of the stability.

    Discussion Questions

    1. Do you have defined test Runs?
    2. What is the worst case for number of times they had to be run?
    3. What is your least number of runs

    Next Week: Process Improvement

  • Test Cycles

    NVP considers a Test Cycle to be one complete execution of a group of test cases. The reason we’re interested in this particular item is that it leads to estimation. The first questions in any testing project are:

    1. How long is it going to take?
    2. How much is it going to cost?
    3. When will you be done?

    These questions can be difficult to answer when starting a project as a new tester or test manager or with limited experience in the software one has been asked to test. Having test cycles helps solve that issue.

    In order to answer those questions we need to:

    1. Define the contents of the group of tests constituting the cycle
    2. Get an estimate of how long each test will take
    3. Add up the resultant times
    4. Build in some contingency
    5. Use that as an estimate for the length of the cycle

    The above gives us an estimate for the length of a single cycle.

    The next question is how many cycles will be run. Our answer is usually three at a minimum on the grounds that there are two debug cycles and hopefully a clean run. In our experience we have managed to get away with two cycles but that’s unusual. Many times it’s many more than three especially if the code is weak or the full requirements are still being worked out. Usually you will have an idea after your first test cycle as to how many will have to be run.

    In order to answer the question of when you will be done, you then need to multiply the number of projected cycles by their individual lengths, add in time for the fixes to be made and promoted and use that as an estimate of the completion date (and the cost by using the chargeback rate).

    1. Do you have defined test cycles?
    2. What is the worst case for number of times they had to be run?
    3. What is your least number of runs

    Next Week: Process Improvement

  • Upcoming Software Testing & Quality Assurance Events – April 2016

    NVP Software Solutions will be participating in the following three software testing and quality assurance events happening this April in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there!

     

    Toronto Association of Systems & Software Quality

    TASSQ – Toronto Association of System and Software Quality – Everything you Wanted to Know about the CSQE! – Brenda Fisk, Director,
    ASQ Canada Deputy Regional Director 2014-2016
    Software Division, Division Executive Team – April 26, 2016 – See http://www.tassq.org/

     

     

    Software Testing in Kitchener Waterloo

     

    KWSQA – Kitchener Waterloo Software Quality Association – the bare minimum you need to know about web application security in 2016 – Ken De Souza – April 27, 2016 – See www.kwsqa.org

     

     MANHATTAN

    London Quality Assurance Peer-to-Peer Contact neil@nvp.ca for more details

  • Testing for System Integrators – Part 5

    Last week our blog discussed the remaining answers to the questions and promised that we would look in detail at two of the answers (which are somewhat similar so we will concentrate on one only).

    There is nothing in the contract (contract is signed) and there is no intention of putting anything in the contract about Quality Assurance.
    Now you have a challenge. Clearly the process is mostly done and there is absolutely no buy-in to Quality Assurance. The next question that needs to be asked is “Why have you brought Quality Assurance in if there is no interest?”

    The key steps here are to determine your position and map out your strategy. There is any number of answers to the question of “Why have you brought Quality Assurance in if there is no interest”.

    1. The final client has belatedly required it. I.e. they have realised it is an omission from the contract and now feel it is incumbent on the System Integrator to provide this as part of the deliverables. You need to determine the final clients needs and work towards those.
    2. The solution is more complex than the System Integrator thought and now they feel a need for Quality Assurance. I.e. like the client above they have realised the value provided by Quality Assurance and now want to implement it even though they were trying to avoid it earlier. There is probably still little buy-in from most of the group. You need to look at each of the Stakeholders and determine their status vis-a-vis Quality Assurance and plan to convert them all to supporters. This is a crucial piece of your strategy in order to be successful.
    3. The System Integrator’s management is becoming nervous and wants Quality Assurance there as a check. While you have management support the team may feel they have an extra burden and possibly someone who is watching them. As in the above, you will need to look at the Stakeholders and see how to convert them to supporters. Otherwise you will get no information at all.
    4. The last possibility is they want someone to blame. This is a tricky one. No matter what you do (either proactively or reactively) they may blame you. You need to plan carefully in order to make sure that your work is recognised as contributing to the success of the project. You need to be very proactive in stating what needs to be done; why it needs to be done and the benefits accruing from having it done. And make sure that everything is documented!

      Happy New year

  • Testing for System Integrators – Part 3

    Over the next few weeks, the NVP blog will focus on Software Testing for System Integrators. From NVP’s point of view, a System Integrator is someone who brings together a number of applications (from vendors), adds some glue and ends up with a solution for the organization they are working with. This seems to agree with the Wikipedia definition fairly closely. So where does Quality Assurance come into this? One would like to think early or very early in the process but that’s not always the case.

    Last week we provided several possible answers to our original

    1. The contract states the following specifically about Quality Assurance and everyone is in agreement
      This means that you simply have to “bridge the gap” between what is expected from the vendors and what is promised to the final client. The only problem may be that you do not agree with the contracted items.
    2. The contract says nothing about Quality Assurance but it’s noted as a topic and the contract will not be finalized without this discussion
      This is almost the best situation. While it may be a little late in the process, the willingness to add Quality Assurance exists and people are behind it.
    3. The contract says nothing about Quality Assurance so far, but now that you have brought it up we will add it.
      The same comment as above is applicable except that there is not quite the backing we might have had earlier.
    4. There is something in the contract about Quality Assurance and we can look it up for you (contracts are signed).
      Well at least they considered it; it may not be correct or complete but it was not entirely ignored. Once you find out what is in the contract you may (or may not) have concerns to handle.
    5. There is nothing in the contract (contract is signed) and there is no intention of putting anything in the contract about Quality Assurance
    6. We don’t know (but that is a good question)
    7. We don’t know (and we don’t care)

    Suffice to say the items in the above list have an obvious gradation from very manageable to a real challenge in the order they are presented. If you get the first answer, you’re well on your way. If you get some of the middle answers you have some work to do, but there’s still time to make change. If you get the last few answers, you are in trouble but not defeated!

    Next Week: What to do with the answers (remainder).