NVP Software Solutions

Category: QA

  • System Boundary Diagram

    System Boundary Diagrams sometimes come up in the context of a Use Case and sometimes in the context of Software Testing. Either way they are a useful in the effort expended when determining what to test. While the ‘normal’ System Boundary Diagram shows the boundaries of the system and thus the boundaries of the testing, we try to use it only as a starting point for other diagrams that may also aid in defining the testing effort and scope. (more…)

  • How Do We Train Software Testers?

    Software tester training is something that appears to require a lot more attention, given the number of conversations we have on a regular basis. We get a surprising number of questions about software tester training and plans, so we’ve dedicated our next blog series on how to train software testers. (more…)

  • Testing Sources of Information

    Sources of information for testing tend to come from two extremes. In some cases you may have almost no information and find it difficult to start. In other cases you may have far too much information and not know what to do with it all or where to start or stop reading. The ‘happy medium’ or ‘just right’ amount is rarely the case when it comes to testing sources of information. (more…)

  • The Out-of-Scope of Testing

    The Out-of-Scope of Testing has only one definition (unlike the Scope of Testing) but it is much harder to document. We want to list everything we are not planning to test! However, the inevitable question is how far to go with our list. Obviously we are not going to list everything we are not going to test and we cannot simply say that everything that is not in-scope is out-of-scope (tempting as that may be).

    The (not entirely rhetorical) questions you will get asked include the following:

    1. Are you planning to omit that from your testing? (should be included)
    2. What made you think that isn’t part of this project? (should also be included)
    3. There is too much risk to avoid testing that.

    The danger with listing out-of-scope items is that they remind people of other items and they then start to think about the risk of avoiding testing that item. You start to get scope creep on your testing and since it is not part of the plan you start to get schedule slippage and cost escalation. Very few people will tell you to test less – it is too risky to state that.

    Whatever your policy; the following will guide the creation of the out-of-scope list:

    1. What can we avoid testing?
    2. What will simply not be able to be completed within budget and resource constraints?
    3. What is the best use of the resources?

    Make sure to document the out-of-scope up front and get it signed off. That will reduce the problems later on and create a much more harmonious working relationship.

    One of the best methods is to build a System Boundary Diagram and let that drive the Scope of testing. Not only does it clarify the thoughts of everyone in the project, it is very effective way of conveying the scope and out-of-scope pieces of the project.

    In general we try to test the interfaces of the System Boundary diagram but avoid full testing of the items outside the System Boundary Diagram. Those are out-of-scope.

    Discussion Questions

    1. Do you define your Out-of-Scope of Testing?
    2. Has it been disputed?
    3. What would you have done differently based on what you know now?

    Next Week: Training

  • Scope of Testing

    The Scope of Testing may relate to how much testing we are going to do or it may relate to how much of the system we are planning to test. The amount of testing could be defined as doing multiple phases of testing with differing aims. The amount of the system we are planning to test and actually do test can be measured by a coverage tool. These two definitions are not independent of each other. Regardless of which definition you decide to use, Be Prepared for some arguing.

    The (not entirely rhetorical) questions you will get asked include the following:

    1. You are planning on how much testing? (too much or too little)
    2. What makes you think that is enough testing? (too little)
    3. You are planning on testing that? (should be included or should not be included)
    4. On what did you base that estimate or expectation?

    The questions can go on like that for ages. I have one client who does not want any errors in production. Their mantra is everything is in scope and test as much as you can. They are no more successful than any one else and sometimes less so.

    Whatever your policy; the following will guide the creation of the scope:

    1. What must be tested?
    2. What can be tested (within budget and time constraints)?
    3. What is the best use of the resources?

    Make sure to document the scope up front and get it signed off. That will reduce the problems later on and create a much more harmonious working relationship.

    Of course, once we have done the scope, then we need to define the Out of Scope. More arguments are on the way! Incidentally Out of Scope is not Everything not In Scope. It must be specified.

    Discussion Questions

    1. Do you define your Scope of Testing?
    2. Has it been disputed?
    3. What would you have done differently based on what you know now?

    Next Week: Training

  • Test Training

    Training seems like an obvious topic and not one to which a blog or two could be usefully devoted. However we get a surprising number of questions about training and plan to address a few of them here. The first one is what type of training is offered. We define three broad categories here:

    1. Training related to testing.
    2. Training related to a particular Test Tool.
    3. Application related Training.

    You only have to read the job advertisements to see the expectations related to open positions. You may see a long list of test tools with which the applicant is to be proficient. You will most likely see some reference to a Test Methodology or SDLC. Most job advertisements finish off with some soft skills.

    So how do our three categories relate to day-to-day work?

    Taking them in reverse order:

    Application related Training

    Clearly the more the person knows about the application for which the system was built, the easier it is to understand the risks, define the scope of testing and explain the results to the business. It is also easier to understand the business requirements and expectations.

    Training related to a particular Test Tool

    This type of training is usually supplied by a vendor and can range from an overview of the test tool allowing one to to use it without in-depth knowledge all the way to becoming a technical expert. The only comment is that every tool has been superceded by something else eventually so every tool or technical process will eventually become redundant.

    Training related to testing

    This type of training covers the rest of the requirements. It teaches about SDLC, Communication, Risk, Planning, and Testing to name only a few items.

    Discussion Questions

    1. Do you participate in Training for Testing?
    2. Was it beneficial to the project?
    3. What would you have done differently based on what you know now?

    Next Week: Sources of Information

  • Test Conditions

    Test Conditions is a term that has multiple definitions. For the sake of this blog, we are going to define them as the equivalent of (Low Level) Test Objectives and state that they are One-Line statements of what is going to be tested. (High level Test Objectives may relate to more system level objectives and some of them may be derived from the Project Charter or plan.)

    For example, the Test Conditions may read as follows:

    1. Test that the system calculates interest correctly.
    2. Verify that the design allows for 100 simultaneous connections.
    3. Validate that the user interface is accessible according to the corporate standards.

    The question that frequently arises is why bother to write these Test Conditions? It seems like an extra step with minimal return. Why not just go directly to the Test Cases?

    We use them for a number of reasons.

    1. They allow the tester to consider the entire system rather than getting into detailed test cases at the first step.
    2. They allow other stakeholders to review the test coverage without having to read through full test cases.
    3. They can identify coverage and omissions in coverage with limited effort.
    4. They allow for estimation of the number of test cases that will be needed before the testcases are written.
    5. They allow for estimation of the test effort early in the project.
    6. They can help identify the components of the test environment earlier allowing it to be specified and built before it is needed.
    7. They determine the required test data and allow it to be gathered and made ready before testing starts.

    We have found that the effort in building test conditions is more than paid back in early information and helpful triggers for what needs to be done.

    Discussion Questions

    1. Do you write Test Conditions or Test Objectives?
    2. Were they beneficial to the project?
    3. What would you have done differently based on what you know now?

    Next Week: Process Improvement – Deal with Results

  • Quality Assurance Process Improvement – Part 4

    Quality Assurance Process Improvement is the current topic for our NVP Blog. We completed a series of 4 blogs on Assessments because at the end of the Assessment process a lot of organizations won’t act on the Assessment results if they don’t have a plan for moving forward. This is particularly true if the Assessment has not been tailored to the particular company in question. A standard Assessment process generates standard recommendations which may not be applicable. Make sure you detail your expectations at the beginning of the Assessment so you get value from the process and your expenditure of time.

    Last blog focused on How to do Process Improvement and now we’ll address “Dealing with the results”.  Many people complete a process improvement assessment; discover a number of problems that need to be fixed but then drop the process without full solving the issues or taking advantage of all of the work that went into getting the results. This typically happens because of the following:

    • The work that needs to be done isn’t scalable or fun.
    • There’s no one to do the work.
    • There’s no budget for the implementation.
    • What was discovered is so unexpected that no one knows how to tackle it.

    These can all be addressed by the ‘divide and conquer’ methodology.

    Once the results of the assessment are known, they need to be organized into logical buckets. Each bucket is then assigned a set of tasks. Some people will tell you that we need to identify the synergies so that everything gets accomplished efficiently with minimal disruption. While that would be the optimal way of doing things; it is rare for anyone to be able to identify all the synergies simply by looking at the list of results of an assessment. We have to accept some redundancy and the fact that some items are going to have to be reversed when new ones are put in place.

    Now is the time to implement your results.

    Next Week: Scope of Testing