NVP Software Solutions will be participating in the following software testing and quality assurance events happening this October in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there! (more…)
Author: Neil
-
A Better Way to Test – 7
A better way to Test – 7
In our last blog we discussed, testing to risk.
One of the methods we mentioned to address Risk was testing, even if it was not the only method. However, stating that we should test and actually completing the task are two different items. Testcases have to be written or at least understood. Results need to be gathered (in some fashion) and reports produced. Whether you are working very formally and need everything recorded or can just rely on what you have seen and experienced without proof does not matter. You will have to do some testing and to do testing you need some sort of statement of what you are going to do and accomplish.
We recommend building test conditions or test objectives in order to determine what is going to be tested. These don’t have to be in detail or have all the components of a standard testcase as long as they are commensurate with the risk of the project. This key point seems to be missed quite frequently. People (in particular those who are under time and budget pressure) want to reduce the level and detail of testing until it does not sufficiently address the risk.
The above reason is why we suggest connecting the testcase to the risk where possible. That makes it much easier to calculate the risk of doing or not doing a particular testcase during a testing cycle. It also, coincidentally, feeds your regression base immediately and allows proper selection when the time runs out.
Take a look at some of the seminars that we offer that address this situation and see if they apply to you. Testing can be better.
Contact us for further information. -
A better way to test – 6
A better way to Test – 6
In our last blog we discussed, as the last point in our list, testing to risk.
If you have the risks listed, explained, classified, calculated, ordered and entered into a database, it may be easy to attach testcases to them and determine when you have addressed the risk with testing. Not all risks can be efficiently addressed by testing, it is simply one of the possible techniques to be used.
However, people rarely agree on the risks and, as mentioned above, not all risks can be addressed by testing. So this needs to be flexible.
If you don’t have your risk recorded then the testcases may still be built to address perceived risks but they may be difficult to justify. Having both sides of the need (the risk and the testcase) can make each justify the other.
Once we have this for one release of the software, the only question is whether we can continue using the same testcases from release to release. Risks will change, some will disappear or be downgraded, new ones will appear or the ones we missed last time will be included for future releases. Like all project requirements, nothing is ever perfect and we progress (sometimes unevenly) to an end result of coverage that is sufficient for the project. Of course something will always surprise us.
The key consideration is not to become inured to the other methods of addressing risk – testing is not the end solution for everything.
Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
Contact us for further information. -
September QA Events – Toronto and GTA
NVP Software Solutions will be participating in the following software testing and quality assurance events happening this September in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there! (more…)
-
A better way to Test – 5
A better way to Test – 5
In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.
One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:
- Requirements (this comes out of Traceability).
- Function tested (may be a subset of the Requirements or it may be a separate mapping).
- Date of testcase creation (just so you know how old it is).
- Priority of the testcase.
- Some statistics generated from the testcase execution.
- The risk it tests – we have rarely seen this explicitly listed.
The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.
When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.
Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
Contact us for further information. -
A better way to Test – 4
A better way to Test – 4
In our last blog we discussed addressing Risk via Testing and one of the problems. Today we want to continue with another of the problems. “More is NOT necessarily Better“. Traceability has come a long way in the last few years. Many of the better test management tools automatically include Traceability modules and generate metrics as a result. The fact that they can measure coverage and provide instant statistics has taken a lot of boring work out of managing a testing effort. Where they can lead astray is in assuming that everything has been checked and that the percentages mean something. We recently visited an Assessment site where the raw figure of 83% of the testcases had been completed successfully in the last cycle. The site was planning on raising that percentage to about 93. Considering where they had started a few years ago; it was a laudable aim and a vast improvement. However, there were several pieces missing from the statistics they were using:
- Whether any of the testcases were redundant and simply repetitions of each other.
- Whether any of the testcases were simply ‘variations on the same thing’. i.e. they were just doing the same thing with slight variations in data
- How long since the last review of the testcases.
- Whether the testcases actually addressed any of the current risks.
Testing Faster or doing More testing may not translate into a reduction in risk. It may actually increase the risk by making people assume that they have done a lot of work and a lot of coverage while completely missing major risks.
Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
Contact us for further information. -
A better way to test – 3
A better way to Test – 3
In our last blog we discussed the determination of risk. The question is what to do with it when we have completed that work. While there is a temptation to write a testcase for every risk; that is not always possible. A testcase or testcases will not (directly) address any of the following issues:
- Schedule issues
- Budget issues
- Poor design
- Poor coding practices
- Incomplete requirements
- etc.
Testing can easily identify, highlight, and quantify some of the above issues. It may bring other issues and risks to light but it will not necessarily resolve or reduce them. The activities to reduce risk need to long precede testing. It is far too late in the project to address risk by traditional testing. Early involvement of Quality Assurance with the emphasis on Process Improvement will go much further to addressing Risk than a few hundred testcases.
All of the above 5 risks which are possible in every project with at least 1,2 and 5 being quite common are all amenable to Process Improvement. Schedule and Budget issues need process work to address the possibilities and see what can be done early. Incomplete Requirements are easily addressed by using a variety of techniques in Requirement Elicitation. We cannot allow ourselves to fall into the trap of assuming a standard method of elicitation (same trap as using standard testing techniques) but we can use a process adapted to the project. The upper levels of Process Maturity are aimed at Innovation not set processes.
Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
Contact us for further information. -
A better way to Test – 2
A better way to Test – 2
The second item to determine is the Risk. We always guess that, if you gather three people from the project together and ask them what the risk is, you will end up with 4 answers: Three from the people at the meeting and the right one that no-one mentions. Levity aside, we do need to figure out the real Risk and it is not always obvious.
Some questions you can ask (and elicit the three answers as a result):- What is the worst thing that could happen?
- Who can be hurt the most if something goes wrong?
- What is going to cost the most if it goes wrong?
Regardless of the answer or answers to the above questions, Quality Control is aimed at Risk Reduction. Everything we do needs to be aimed at reducing the risk in the project. Doing that can mean a lot of things that are far beyond traditional testing. We might want to ask what could be done now (before the end of the project) to reduce the risk. We might want to go outside the standard boundaries of the SDLC to consider solutions. This comes back to the need for early involvement. When the project is too far along, it is difficult to suggest radical solutions that might be beneficial. People are too set in their ways and the project is careening to an end that no one likes.
This is why people who advocate a standard methodology or approach to testing for every project are locked into something that may not work. The same solution does not always work twice!
Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
Contact us for further information.