NVP Software Solutions

Category: Software Testing

  • A Better Way – Case Study 2 – Thinking Like the Client

    A Better Way – Case Study 2 – Thinking Like the Client

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The second case study might be called “Thinking Like the Client”.

    Even though many in IT have embraced Agile in some form or another and with some level of success, not all clients are so willing. We find this particularly the case with larger, older and more safety or image conscious clients. They are not willing to take the risk of everything not coming together at the correct time. They are not willing to live with constant change. In addition, they often have milestones that need to be fulfilled and quite frequently they have stated contract deliverables that need to be fulfilled so that payments can be made.

    We received a call from the President of a development company with a large contract with probably the oldest, largest and most image conscious client you could get – the federal government. Even though the company had delivered projects successfully before and had a good record of coming through with what was required, the client still wanted a formal Test Plan completed.

    In our experience, the client is normally okay with a High Level Master Test Plan provided it contains the following (this is not an exhaustive list by any means):

    1. High Level Objectives
    2. A listing of the types of testing needed and the risk around each type
    3. Processes for Test Creation and signoff
    4. Processes for Defect tracking and resolution
    5. Methodologies for executing the testcases

    This reassures them that the developers have an understanding of what needs to be done.

    If you want to discuss this further contact us.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A Better Way – Case Study 1 – Thinking Outside the Box

    A Better Way – Case Study 1 – Thinking Outside the Box

    In our last several blogs we have discussed ‘A Better Way to Test”.

    The issue is to apply this to actual situations. We have 5 Case Studies we plan to use over the next several weeks to address this. The first one revolves around a junior QA, a very successful (small) company and a need to test effectively and quickly for a large final client. The issue that came up was the ability to “Think outside the box”.

    We received a call from the President of the company indicating that the new junior QA was having trouble considering things “Outside the box”. They were good with what was presented in the Use Cases or User Stories. Most people can generate testcases based on what is provided and the happy path from the Use Cases and User Stories. Experienced testers will apply other techniques and may specifically try Exploratory testing. However, you cannot explore or test what you don’t know or don’t think of at the time.

    Rather than dictating “Outside the box” which is a contradiction in itself, we decided to go with more of a checklist approach listing some of the areas that would be considered to be non-Happy Path and see if it could lead to further extensions. We did not dictate everything but started with more of a charter and guidance list to see what would come of it. The process had two advantages:

    1. We would have some coverage of everything “inside” and “outside” the box.
    2. We could use the results to evaluate the suitability of the junior QA for further roles and projects.

    The President came up with the basic list and ran it from there.

    If you want to discuss this further contact us.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A Better Way to Test – 7

    A better way to Test – 7

    In our last blog we discussed, testing to risk.

    One of the methods we mentioned to address Risk was testing, even if it was not the only method. However, stating that we should test and actually completing the task are two different items. Testcases have to be written or at least understood. Results need to be gathered (in some fashion) and reports produced. Whether you are working very formally and need everything recorded or can just rely on what you have seen and experienced without proof does not matter. You will have to do some testing and to do testing you need some sort of statement of what you are going to do and accomplish.

    We recommend building test conditions or test objectives in order to determine what is going to be tested. These don’t have to be in detail or have all the components of a standard testcase as long as they are commensurate with the risk of the project. This key point seems to be missed quite frequently. People (in particular those who are under time and budget pressure) want to reduce the level and detail of testing until it does not sufficiently address the risk.

    The above reason is why we suggest connecting the testcase to the risk where possible. That makes it much easier to calculate the risk of doing or not doing a particular testcase during a testing cycle. It also, coincidentally, feeds your regression base immediately and allows proper selection when the time runs out.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
    Contact us for further information.

  • A better way to Test – 5

    A better way to Test – 5

    In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.

    One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:

    1. Requirements (this comes out of Traceability).
    2. Function tested (may be a subset of the Requirements or it may be a separate mapping).
    3. Date of testcase creation (just so you know how old it is).
    4. Priority of the testcase.
    5. Some statistics generated from the testcase execution.
    6. The risk it tests – we have rarely seen this explicitly listed.

    The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.

    When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to test – 3

    A better way to Test – 3

    In our last blog we discussed the determination of risk. The question is what to do with it when we have completed that work. While there is a temptation to write a testcase for every risk; that is not always possible. A testcase or testcases will not (directly) address any of the following issues:

    1. Schedule issues
    2. Budget issues
    3. Poor design
    4. Poor coding practices
    5. Incomplete requirements
    6. etc.

    Testing can easily identify, highlight, and quantify some of the above issues. It may bring other issues and risks to light but it will not necessarily resolve or reduce them. The activities to reduce risk need to long precede testing. It is far too late in the project to address risk by traditional testing. Early involvement of Quality Assurance with the emphasis on Process Improvement will go much further to addressing Risk than a few hundred testcases.

    All of the above 5 risks which are possible in every project with at least 1,2 and 5 being quite common are all amenable to Process Improvement. Schedule and Budget issues need process work to address the possibilities and see what can be done early. Incomplete Requirements are easily addressed by using a variety of techniques in Requirement Elicitation. We cannot allow ourselves to fall into the trap of assuming a standard method of elicitation (same trap as using standard testing techniques) but we can use a process adapted to the project. The upper levels of Process Maturity are aimed at Innovation not set processes.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 2

    A better way to Test – 2

    The second item to determine is the Risk. We always guess that, if you gather three people from the project together and ask them what the risk is, you will end up with 4 answers: Three from the people at the meeting and the right one that no-one mentions. Levity aside, we do need to figure out the real Risk and it is not always obvious.
    Some questions you can ask (and elicit the three answers as a result):

    1. What is the worst thing that could happen?
    2. Who can be hurt the most if something goes wrong?
    3. What is going to cost the most if it goes wrong?

    Regardless of the answer or answers to the above questions, Quality Control is aimed at Risk Reduction. Everything we do needs to be aimed at reducing the risk in the project. Doing that can mean a lot of things that are far beyond traditional testing. We might want to ask what could be done now (before the end of the project) to reduce the risk. We might want to go outside the standard boundaries of the SDLC to consider solutions. This comes back to the need for early involvement. When the project is too far along, it is difficult to suggest radical solutions that might be beneficial. People are too set in their ways and the project is careening to an end that no one likes.

    This is why people who advocate a standard methodology or approach to testing for every project are locked into something that may not work. The same solution does not always work twice!

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • A better way to Test – 1

    A better way to Test – 1

    The first thing to determine is the type of project, type of methodology being used to build it and the people involved (All of them) in building the project. This is not new by any means and seems obvious information to gather, but in the rush to get at testing when you have been recruited late in the project, this is often either skipped or compressed (usually with dire results later on). A few leading questions or a template that needs to be filled in for the Project Test Plan can often help in ensuring that you get all the information that is available. Frequently we find that this also helps the other people in the project get a better understanding of what they are dealing with and some of the expectations.

    We started into this process at a recent project and elicited the ‘surprising’ information that the design was wrong. The vendor’s exact words were ‘That is not the way our product works’. The project ran a lot longer than was expected and required a lot more testing since the design was revamped at great cost late in the project and the changes had to ripple throughout the software and hardware.

    The above is simply preliminary information that needs to be gathered before we even consider what we are going to do with it and how it impacts our testing.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • Isn’t there a better way to Test?

    Isn’t there a better way to Test?

    This is a question that gets asked very frequently especially at the end of projects that have not gone well. Testing is often left to the end of the project and by the time people get to it they are tired of the project, tired of the project team and generally looking for something new. Even when projects do go well; people ask the same question.

    The answer of course is “YES”. Testing can go better. But it requires a shift in the way it is done. Some people have a set methodology they use for all testing projects. Others rely on specific tools that are supposed to aid them in testing properly. Others work out what they are going to do when they get to the project. If every project was identical and things had been successful in the past then these approaches will work. However, no two projects are alike and the requirements for testing or Quality Assurance differ from project to project.

    Sometimes the differences are minor and techniques and processes used in the past can be reused with slight modifications. However, even with minor differences, testing needs to evolve over time to take into account new technologies and new methodologies. With major differences in projects and expectations much more than just a slight evolution must be undertaken. This will be the subject of the next few blogs.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.