NVP Software Solutions

Category: QA

  • How interactive prototyping can improve QA in the SDLC

    It’s often said that quality must be built in, not added on. But when it comes to the Software Development Lifecycle (SDLC), the reverse often happens: defects are identified late on in the Testing Phase, after coding is done. This means bugs are expensive to fix and solutions are found last-minute, putting quality at risk. Early Lifecycle QA, from requirements definition onward, results in a better software development experience and, hopefully, a better end product.

    But even when Early Lifecycle QA does happen, it’s not always plain sailing: business requirements documents are often scanty and don’t provide QA professionals with enough information; other stakeholders may be resistant to QA specialists coming in and “telling them their job” at the review stage; some requirements are untestable thanks to lack of clarity. And of course things change throughout any project, it’s a fact. Flexibility is a must.

    So how can QA professionals ensure that they can get involved and be effective from the outset of the SDLC and throughout it? Step up interactive prototyping. Using an interactive prototyping tool can facilitate early stage QA and avoid common pain points.

    Requirements definition and gathering

    QA specialists sometimes receive little information on which to base tests at this stage, thanks to paltry requirements or incomprehensible Business Requirements Documentation (BRD). Additionally, QAs are often sent the documentation too late, meaning there’s no time to set up adequate tests. By gathering, defining and gathering requirements using a prototyping tool – requirements can be imported or created directly in the prototype, and all invited stakeholders (including QAs) can add or comment upon those requirements in real-time. Once you have the baseline of requirements, a System Testing Plan can be finalized.

    Interactive requirements and iterative process

    Once the BRD and System Requirements Specification are agreed upon, the QA team can set about reviewing requirements in the prototype. Running user test cases with a designated User Proxy – someone who takes on the role of User – will allow QA to be approached from 3 angles: functional, structural and conformance. All QA team members can add to and edit the BRD in the prototype, ensuring that user and system needs are accurately represented at this early stage.

    Using a prototyping tool to facilitate this process reduces time and budget concerns for project managers, which means they are more likely to agree to incorporating QA teams early on.

    Design and QA

    With a version history of requirements accessible within the prototype, the design team has a clear map to work off. They can build an interactive prototype based on the validated requirements, linking each feature to its relevant requirement and thereby facilitating QA testing. Once the design team has produced a high fidelity prototype, activities such as verifying system architecture and carrying out system audits can be done on the prototype. Finding and fixing bugs through prototype testing is a lot cheaper than fixing them in the code.

    Coding and Deployment

    Later SDLC stages can now go ahead, with the QA team carrying out coding-related Quality Assurance activities such as verifying implementation of top requirements, and checking the quality of code with Product Quality Analyzer tools.

    Key Success Markers

    Early Lifecycle Quality Assurance requires collaboration between teams and a shared vision, factors supported by the inclusion of interactive prototyping in the SDLC. By prioritizing Early Lifecycle QA rework and costs are reduced, QA input is incorporated at every stage of the project, and time to market is optimized.

    Justinmind is a prototyping tool for web and mobile applications that allows you to visualize your software solution before starting development

  • Quality Assurance Process Improvement – Part 3

    Quality Assurance Process Improvement is the current topic in our Blog Series. We completed a series of 4 blogs on Assessments because at the end of the Assessment process a lot of organizations won’t act on the Assessment results if they don’t have a plan for moving forward. This is particularly true if the Assessment has not been tailored to the particular company in question. A standardized Assessment process generates standard recommendations which may not be applicable. Make sure you detail your expectations at the beginning of the Assessment so you get value from the process and your expenditure of time.

    Last time we looked at Why Carry Out Process Improvement and now want to address the question of “How to do it”. First we need to identify the processes that we want to improve. We will assume that this step has been completed already. Then we need to measure the existing process for what we want to improve. If we want it to be faster then we need to measure the time it takes. If we want it to be more consistent, then we need the measure the output against some standard. Once we have a baseline of measurements (you will probably be measuring more than one item since we do not want to improve one at the expense of another), then we can decide how we want to improve it. If we want the process to be faster, we might try to get the inputs to it at the right time or before they are needed. We might try cutting out any extra steps. While doing this we do not want to reduce the quality of the output so we will want to measure that as well. if you want it to be more consistent you might look for places where the product deviates from standard and try to improve those while making sure that does not make any other pieces of the product worse.

    The post improvement measurements should always be carried out after the process has had a chance to settle down and achieve some stability. Measuring too soon may lead to erroneous conclusions.

    The objective is to save the overall organization funds; not just the current project. As such the results of an Assessment and the activities done as part of Process Improvement must be assessed at the corporate level and not at the project level.

    Next Week: Guest Blog

  • Scheduling Test Cycles

    Scheduling Test Cycles often seems to create challenges for Managers, so we thought we’d tackle this for today’s blog. In our experience, there seems to be an ingrained view from Test Managers and Development Managers to not leave time between the Test Cycles or the Fix Cycles for the other party to do their work.

    I have seen Test Cycles scheduled consecutively with no room to actually fix anything. The idea was that they could fix the bug overnight or during the weekend because nothing could impede the test effort at this stage. The alternate problem is scheduling by the Development Manager who puts all the time into the Fix or upgrade time and allocates nothing for testing. The same question elicits a similar answer that testing can proceed overnight or on weekends.

    What is obvious is that there has to be compromise on both sides.

    However, it is possible to schedule overlap. Certainly Developers can start fixing bugs found early in the test cycle. before the cycle is finished, and it’s probably better that they do. However, this requires strong promotion and code control procedures and a plan on how the environments are going to organize. Otherwise, fixes start getting into the test environment before other testing is done. Similarly testing can continue even when Developers are in fix-mode. Planning is required to cover items they aren’t working on at the moment.

    We just need to plan our way through this with the understanding that there will be changes as the project evolves, dependencies arise, and items change.

    Discussion Questions

    1. Have you Scheduled Test Cycles?
    2. If yes, to number 1, how did it work out?
    3. What would you have done differently based on what you know now?

    Next Week: Final meetings for the year

  • Quality Assurance Process Improvement – Part 2

    Quality Assurance Process Improvement is the current topic in our Blog Series. We completed a series of 4 on Assessments because at the end of the Assessment process a lot of organizations won’t act on the Assessment results if they don’t have a plan for moving forward. This is particularly true if the Assessment has not been tailored to the particular company in question. A standardized Assessment process generates standard recommendations which may not be applicable. Make sure you detail your expectations at the beginning of the Assessment so you get value from the process and your expenditure of time.

    Last time we looked at What Process Improvement is and now want to address the question of “Why do it”. We stated earlier that we needed to understand the intent of an Assessment and use it going forward. And that is  after-the-fact in terms of the answering the question. Why do an Assessment and continue on with Process Improvement in a Quality Assurance environment in the first place?

    The answer is that Process Improvement Saves Time and Money. We do not carry out any Process Improvement activity without the intent of saving money. The activities we do must have a positive ROI. This, however, is the more difficult question to answer since the positive ROI is not always in the current project. Putting in a defect management process; improving the review process; ensuring early involvement of Quality Assurance personnel in a project benefit the next project but not the current one.

    The objective is to save the overall organisation funds; not just the current project. As such the results of an Assessment and the activities done as part of Process Improvement must be assessed at the corporate level and not at the project level.

    Next Week: Scheduling Test Cycles

  • Test Run

    Our latest blog will discuss the Test Run. For today’s purpose, NVP considers a Test Run to be one, single execution of a testcase. This could mean that the testcase ran to completion and the expected AND actual results were identical, or that the case the testcase did not have actual results that equalled the expected. We have stayed away from the words ‘successful’ and ‘unsuccessful’ since some may feel a testcase is only successful if it uncovers a problem and is unsuccessful if it does not.

    We are interested in this statistic of test runs for a number of reasons:

    1. It helps in estimation
    2. It helps justify the time taken to test
    3. It provides a measure of code stability

    Estimation

    Knowing the number of Runs of a testcase helps determines how long the cycles and the whole test effort will take next time. If we know we had to run each testcase an average of 5 or 6 times before it ran to completion without raising an issue then we know how many times we may need to run it next time. Note that unsuccessful runs may include attempts that lead to fixing the testcase or relevant test data. Once we have ‘debugged’ the testcase, these runs may not recur.

    Justification

    If we only report the count of completed testcases with actual results equalling expected results, then each testcase might only show a single execution. This would hide a lot of work and effort and make the testers appear very unproductive. Showing that each testcase was executed 6 or 7 times before we were satisfied gives a much better idea of the effort involved.

    Code Stability

    If a testcase is run a dozen times and only on the last time does it run to completion with Expected Results equal to Actual Results, then we may have a concern with code stability or whether that final run was really correct. Something that fails a dozen times and then is successful is highly suspect. Maybe the conditions changed, maybe we missed something, maybe the issue was finally fixed. Whatever the case, we are not sure of the stability.

    Discussion Questions

    1. Do you have defined test Runs?
    2. What is the worst case for number of times they had to be run?
    3. What is your least number of runs

    Next Week: Process Improvement

  • Quality Assurance Process Improvement – Part 1

    Quality Assurance Process Improvement is the next topic in our Blog Series. We completed a series of 4 on Assessments and at the end of the Assessment process a lot of organizations tend not to act on the results if they do not have a plan for moving forward. This is particularly true if the Assessment has not been tailored to the particular company. A standardised Assessment process generates standard recommendations which may not be applicable. Make sure you detail your expectations at the beginning of the Assessment so you get value from the process and your expenditure of time.

    Assuming that you have the results of an Assessment available, the next step is often some sort of Process Improvement. So what is Process Improvement?

    We define Process Improvement as a planned and measured activity that leads to a better process.

    Planned – Process Improvement carried out without a plan is simply change for the sake of change with no knowledge of whether this is an improvement or not.

    Measured – Measurements need to be taken both before and after the process improvement activity is completed. Before measurements provide a current baseline of how the process is currently operating. After measurements determine whether the process improvement activity has had the desired effect. Without both, there is no point to the process improvement.

    Better Process – The measurement process tells us whether the process has improved at all. There is a number of things that could be measured. Maybe the process has got quicker; maybe it is cheaper in some way; maybe it is more consistent so the results are more similar; or maybe the end product or service generated from the process is better. It would be nice to get all four with one process improvement activity but frequently it will require multiple iterations of the process improvement cycle to start realising gains on all four items.

    Next Week: Coming Meetings

  • Test Cycles

    NVP considers a Test Cycle to be one complete execution of a group of test cases. The reason we’re interested in this particular item is that it leads to estimation. The first questions in any testing project are:

    1. How long is it going to take?
    2. How much is it going to cost?
    3. When will you be done?

    These questions can be difficult to answer when starting a project as a new tester or test manager or with limited experience in the software one has been asked to test. Having test cycles helps solve that issue.

    In order to answer those questions we need to:

    1. Define the contents of the group of tests constituting the cycle
    2. Get an estimate of how long each test will take
    3. Add up the resultant times
    4. Build in some contingency
    5. Use that as an estimate for the length of the cycle

    The above gives us an estimate for the length of a single cycle.

    The next question is how many cycles will be run. Our answer is usually three at a minimum on the grounds that there are two debug cycles and hopefully a clean run. In our experience we have managed to get away with two cycles but that’s unusual. Many times it’s many more than three especially if the code is weak or the full requirements are still being worked out. Usually you will have an idea after your first test cycle as to how many will have to be run.

    In order to answer the question of when you will be done, you then need to multiply the number of projected cycles by their individual lengths, add in time for the fixes to be made and promoted and use that as an estimate of the completion date (and the cost by using the chargeback rate).

    1. Do you have defined test cycles?
    2. What is the worst case for number of times they had to be run?
    3. What is your least number of runs

    Next Week: Process Improvement

  • Quality Assurance Assessments – Part 4

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal. This week we will discuss What to do with the results of a QA Assessment now that we have completed an explanation of HOW to do Assessment in a past blog.

    What to with the results of a QA Assessment

    There is a strong temptation to (facetiously) say Do Nothing with the Results since that happens so frequently. The Assessment is completed and everyone just wants to forget about it. Not only is that a direct waste of the effort and time included in the assessment, it also sends the signal to everyone that their effort was unnecessary and their thoughts unappreciated. Don’t expect a lot of effort next time under this scenario.

    If we use the example from the last blog (referenced above) of the questionnaire or in person interviews to elicit the information using open-ended questions, then we will end up with a lot of disparate information that may not be readily parsed.

    The steps are as follows:

    1. Review all the provided answers.
    2. During the review write down some general categories for the answers (i.e. insufficient testing; requirements issues; development issues; testing issues). If these categories were predetermined then this step does not apply.
    3. Allocate the answers into the categories.
    4. Allocate the answers that fit into more than one category (put them into both).
    5. Allocate the answers that only occur once and do not fit into any category (make a category of Other and put them there).
    6. Extract a common consensus from each category (there is a lot of work in this step)
    7. Start a process of finding the root cause of the common problems.

    Now we have to act on the root causes and resolve them. This could be a whole series of blogs but we will leave that for the process improvement cycle.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. Has anyone acted on the results?
    3. Were the results used for Process Improvement?

    Next Week: Vocabulary