NVP Software Solutions

Category: Uncategorized

  • Factory Testing

    Factory Testing is one the terms that leads to a lot of acrimony in the industry. There are people who believe it and will evangelize for it and others who claim it stifles every innovative thought ever made.

    Some of the people who believe in it fervently are those who have exacting expectations from their clients. They need to be able to tell them with certainty what tests were executed, when they were executed and what the results from those tests were. At the end of test, there is no question as to what has been done and the decision about whether to proceed is easy.
    Those who believe that this process stifles creativity think that simply following fixed scripts without thought reduces the role of a tester to that of a checker and not a tester and the work might as well be done by someone without a testing background.

    There is room for both views but they need to be applied at different points of the Software Development Life Cycle.

    Creativity needs to applied (in both views) at the time of Test Case or Test Objective Creation. We certainly need creativity and original thought in determining what to test and how to test it. Once that has been done and the final client has agreed that the coverage is sufficient and then the paths diverge a little, but not as much as some people think.

    In the Factory Testing method, the testers (or checkers) must finish all the tests that are approved in the way they have been written. This does not preclude them discovering other ways of testing and new tests that can and should be completed. it just means that they may not necessarily complete them immediately for the current cycle or release. They certainly should not be lost and can be used to improve the coverage and process next cycle or release or even in the current period if they are important enough.

    In the other methodologies, we can still use the approved tests as a starting point and then start to add and expand. However, we do need to know the best way of augmenting tests while ensuring coverage. Quality Assurances focus on statistical analysis will allow the calculation of those figures and make that decision.

  • Early vs Late Testing

    Early versus Late Testing within a project is an ongoing debate within the software testing industry. With the Waterfall methodology, active testing was not possible until late in the development cycle when the project was ‘thrown over the wall’ to testing and the testers started what some called the ‘final exam’. A long test-fix cycle often ensued. Many newer methodologies promote earlier involvement and many of the recent ones promote continuous testing of multiple builds using test automation. With a broad definition of testing that encompasses more than just direct Validation it is quite possible to start some form of testing very early in the project. However, some feel that we have obtained as many benefits as we possibly can from early testing and we need to bring back an emphasis on late cycle testing when the system is fully developed in order to prove everything works together.

    This debate is missing the point of the testing process. The only question is when is it appropriate to do testing and this can be answered at the time of planning and with the help of Quality Assurance. Testing has to be completed when it is appropriate and when the necessary items are in place to support that endeavor. During the test planning process, it is necessary to identify what testing needs to be completed. Once we know what has to be completed, then we can identify where in the process it can be completed. Quality Assurance processes allow that identification to take place easily. With some overarching process in place that indicate what the organisation requires in the way of Risk reduction, it is easy to identify what testing is required and then map that to the appropriate place in the Software Development Life Cycle.

    While Early or Late testing may not always be appropriate, early planning is always necessary and Quality Assurance guides that planning.

     

  • The army never retreated; they simply turned around

    The full quote that I recall is that “The army never retreated; they simply turned around and advanced in the opposite direction”. (However, I cannot find this quote online.)

    Many software projects try different paths and different solutions for problems encountered and some of the newer development methodologies are built for exactly this type of trial and error process and work much better than the older methodologies in that respect. If there is a problem or something is not quite what is desired, then the ability to switch it at the last minute and redo it is built into the process. With older methodologies one was locked into the solution very early in the process and it was very difficult and expensive to switch after the start. Flexibility is required as is the ability to change direction mid-stream as the business conditions change. Some people believe that Quality Assurance mandates against this type of flexibility (and believes that the advance always has to be in the forward direction), however, as we will see below, that is not the case.

    The above quote implies a complete turn around with a complete change of plan and in all aspects of the project. While that may be necessary if the project has been incorrectly designed and created from the beginning; it is also very expensive. A complete reversal of the project implies discarding all that has been done or gained (much like the reversal of the forward motion of the army) and a restart somewhere further back in the process with a loss in time and an expenditure of resources that is now lost.

    The Process Improvement aspect of Quality Assurance addresses this concern. The need for flexibility is built right into the process. The understanding that change is inevitable and the ability to not only embrace but facilitate that change is included in the defined processes and made part of them. Furthermore Quality Assurance processes are subject to review on a regular basis in to ensure that they have not become outdated or are currently impeding process.

  • DevOps and Quality Assurance

     

    A recent article in Techwell highlighted two interesting points for DevOps and Quality Assurance.

    The main problems seem to be included in the following paragraph from the article:

    ‘One problem with this approach is that the business sometimes cannot handle changes so rapidly. The other problem is that too often, the QA and testing team cannot keep up with the need to verify and validate changes. DevOps may be able to deliver changes to production, but the rest of the team may not be able to handle continuous deployments. Improving the QA and testing function can help deal with these challenges and enable organizations to effectively handle the impact of continuous delivery and continuous deployment.’

    The first question is in the initial sentence. Can the business handle the rapid changes? The better question to ask is whether the business actually wants the rapid changes in the first place. If they express the need for the changes and agree to the timetable for their delivery then they would be expected to put the resources and allocate the time to learn and utlise the changes as they arrive. An assessment of the ability of the business to absorb the rapid pace of change and a subsequent prioritisation of the order of changes would reduce this problem substantially.

    If the business expresses the desire for all the changes and states that the delivery schedule is necessary then we come to the second problem – the ability of Quality Assurance and Software Testing to deliver. As a recent comment on LinkedIn put it “Are you sure it is testing you are doing” or it might be better worded as “Are you sure it is testing you should be doing?”. It might be better to check items earlier and determine what is necessary rather than assuming that Testing is all that is required. Quality Assurance carries out that assessment.

  • Big Data – How Not Why

    A recent talk on Big Data illustrated one of the conundrums that seems to be currently plaguing the industry. The talk addressed HOW to deal with Big Data assuming that it had already been gathered but not WHY it was gathered in the first place. Fortunately an Economist article from last week “Off the Map” provided an excellent answer to the question of why collect it for a certain reason, but not in general, as a best practice. This questionable practice is similar to that of some of the past fads that have come through the IT industry. We spend a lot of time stating how to do do something without really asking why it is being done. Automated testing tools are one example as are some of the testing methodologies that sweep through from time to time. Another one is a methodology that promises to do testing better or faster as long as it is worth doing. The worst (but simplest) example of over-gathering data was a QA Manager at an insurance company who insisted on something in the order of 50 mandatory fields to be filled out for each defect raised. Few of these defaulted and most had to be selected from lists. Certainly the mass of generated data could be mined in many different ways but the impact on defect creation was substantial and lead to the usual attempts to bypass the system.

    In the case of Big Data, some of the attitude is to collect it ‘because we can’ with no necessary end view in mind. Quality Assurance always takes a more sustainable view and confirms the reasons for any particular action before embedding it into general use. Several weeks ago we mentioned how Quality Assurance supported Green Initiatives. This is directly in line with that sentiment. We do not pile up statistics or data without a reason for its need and a planned usage.

  • Startups and Small Companies Need QA

    There is a common view that Quality Assurance is restricted to larger companies like Banks, Utilities, and Telecommunications organisations. This view is misleading: Startups, Small Companies and Not-for-profit groups also need QA. As a matter of fact, in many ways they need it more than some of the larger companies because larger companies have the resources to recover from a problem and often have the reputation to ride out many issues that could flatten a small company.  A Startup or Small Business is often riding on limited dollars, time and staff and does not have the resources to recover from a large problem that could affect their product or their reputation.

    Quality Assurance that a smaller organisation needs does differ from that of a larger organisation. A large organisation requires Quality Assurance processes and procedures that will allow it to continue in business while minimizing the risk to their existing systems. A smaller organisation requires Quality Assurance processes and procedures that allow it to react rapidly to market requirements and grow quickly if necessary. A small organisation does not usually need extensive Quality Control or Software Testing – they know their product inside and out and and can quickly make changes as required. Startups or small businesses need Quality Assurance that ensures that any changes made fall directly in-line with the long term growth plan of the company and the product. This might involve some of the following:

    • Design considerations for the product to allow for future growth
    • Guidelines on how enhancements are going to be added to the product so that the overall product direction is not lost
    • Coding standards to ensure future flexibility of growth

    None of the above require a huge investment in time or resources (both of which are usually in short supply in a new company). They just require some consideration and the discipline to go back every few months and make sure they are effective and efficient.

     

  • Apps Need Quality Assurance Also

    Many people equate the requirement for Quality Assurance with old applications in large businesses. However Apps need Quality Assurance also, although for different reasons.

    Many of the old applications in large businesses are mission critical to the business and their profitability. Failure of those applications will have major repercussions and may drive the business into bankruptcy. The traditional argument for Quality Assurance and Quality Control hinged on that concern and there is no reason to change that attitude in that situation. As a colleague said, ‘I need to know what was done, how it was done and I need to know it was done correctly. Then I can go to my management with a recommendation that the application be released into production with assurance that it will work’ Suffice to say he works for a bank and a failure will hurt.

     

    At the other end there are the thousands of individual apps now available on a series of platforms and operating systems that are targeted at a specific limited audience and may have limited appeal to the bigger audience. The comment I heard at a recent TASSQ meeting is that the Apps must ‘Excite’ the audience and furthermore they must get good reviews, no negative reviews, and be somewhere close to the top of the popularity list. Otherwise you might as well withdraw, rebuild, rebrand and re-release the app as new. It is too late to fix after the fact. If that sounds suddenly familiar that is because we have just repeated the Quality Assurance argument. More than ever we need to get it right the first time. As stated above, there is now no time to fix it. We only have one chance. This does not mean bogging down the app in a long drawn out testing cycle. This means getting it right all the way along the cycle. Then we will have apps that excite! Quality Assurance provides that guidance.

  • Sometimes Simpler is Better!

    We have all heard that ‘Simpler is Better’ and a recent personal experience proves the point. Two members of my family flew to Thunder Bay – one went to write an exam and the other one went along for the ride. They were to return on Saturday after the exam. They arrived at the Thunder Bay airport for the return flight with time to spare and started waiting for the plane. At the other end of the flight (Toronto) I was watching the arrivals online so I would know when I had to go and pick them up from the airport.

    The time for the flight to depart from Thunder Bay got closer and closer and then eventually passed with no sign of a plane. What was interesting was the following:

    • The Departures Board still indicated the flight was leaving on time
    • There was no one at the airline check-in desk to provide any updated information
    • At the other end, I could see on the airline’s website that the flight was delayed

    Eventually some of the passengers asked the security guard if he knew what had happened to the plane. He looked out the window at the gate and stated the obvious: No plane is here yet so it is late. Sometimes Simpler is Better when it comes to answering a question. They could have signed onto the airline’s site and would have seen the updated status but it is expected that would be reflected in the local information.

    This same issue appears sometimes in Quality Assurance. Some people are under the impression that Quality Assurance is trying to make things harder. This is the exact opposite of what it is trying to do. Quality Assurance makes processes and work simpler and allows more time for the creative problem solving side. Processes or items that are standard are made into a process. Items that require on-going problem solving skills now have more available resources. Visit how Quality Assurance can make you more creative.