Blog

Many consultants will provide the Standards (what needs to be done) but not the process to be followed. People have their list or toolbox of of standards that they apply in every engagement. At the end a report is provided saying what standards need to be achieved but rarely is a process road map provided that allows the organisation to achieve those goals. Standards without processes to achieve them are not of any use to an organisation already running at high speed in an attempt to keep ahead of the competition. What is even more critical is the fact that the process is never finished. Once the new level of standards are achieved (which takes time) then there needs to be a review process built in to ensure continued improvement over time (Deming Wheel or Shewhart Cycle).

Two other key considerations are to ensure that we are really fixing on-going problems and not just reacting to the inherent system variability. It is very easy to fall into the trap of simply reacting to every little change and trying to fix that. Doing that leads to more instability as we adjust the system back and forth chasing the process that will provide perfect stability while simply adding more wild swings to the result. It is very similar to oversteering when the car is sliding on ice. Minor changes in the steering result in a controlled car moving in a straight line. Major corrections usually result in the ditch!

The other major item to watch is the one time causes of issues or those external to the system that impact it but are rare and or out of our control. Trying to build solutions for these into the system adds unnecessarily to the complication and may never address the next one time error that occurs. These need to be addressed by contingency plans that can be implemented when necessary but are not otherwise used.

Make sure you are concentrating on the critical issues; make sure you have a process for getting to where you want and then see if you have attained the standard you want.

Recently a talk was given at a local association and clearly the presenter felt a need to be dramatic (at the very least). The initial statement was that Regression Testing was bad now and would only get worse. Under two current methodologies (listed below) that is the case:
1. Continually add to the regression test base and retain all that is there. The ‘more is better’ approach. Eventually the regression base grows too large (as happened to a client of ours where the regression test for the release every quarter occupied several people testing for the entire quarter and took more than a year of resource time to complete) and thus takes an increasing share of the testing budget to the exclusion of useful testing.
2. Ditch the entire base when it is felt that it is no longer providing any benefit (has not found anything new for several releases) and either start with a new base or forget about regression testing.

Either of the above approaches are expensive in either resources or risk – take your pick.

The correct way is to make sure that the regression is planned and organised to serve its purpose which is Risk Reduction commensurate with outlay.

Recently we found ourselves unexpectedly with a group of people who had been told what they needed without being asked what the fundamental problem was or how it could be solved. No needs assessment had occurred. Instead a quick solution had been sold to them and we had been asked to deliver it on behalf of someone else. A brief conversation at the beginning of the meeting elicited that the requirements were completely different and needed to be delivered in a different way. It pays to know the needs first.

One of the concerns with Process Improvement is the tendency to throw out all the old processes and start anew as if nothing had ever existed. This is an understandable reaction if the existing processes have never been organised, recorded, and classified. The problem is compounded if they are scattered throughout the organisation, exist only in unwritten format, and vary from department to department. However, the effort to create net new processes far exceeds that of recording the old ones. It is worthwhile to inventory the existing processes first before emabarking on wholesale creation.

“Bouncebacks’ is not a technical term in common use. In our usage here we are counting the number of times a defect (or issue) bounces back and forth between tester and developer (or other personnel). It is common for a defect to have a count of at least 2 and maybe 4 if some clarification is required. Here we are looking for defects or issues that are bouncing back and forth between development and testers too many times. One of our clients recently did this and then concentrated on the items that had high counts. Not surprisingly they discovered that the majority of the issues with high bounceback counts revolved around poorly written requirements. Instead of getting the requirement correct in the first place, the concept was being clarified by continuous back and forth communication late in the project with a high overhead and testing cost. Call us for more information.

The demographic dividend occurs when fertility rates drop; and and a bulge of people move through the society creating a reduced dependency ratio. A definition can be found at http://en.wikipedia.org/wiki/Demographic_dividend. You might ask what impact this has on testing. Several countries (in the world) are currently experiencing a demographic dividend right now. Some are just entering it, other are in the middle and some are exiting it very quickly (more quickly than they would like). With Quality Control (Software Testing) being a worldwide industry, it has been easy in the past few years to send the testing where the population is booming. This has kept costs low and reduced pressure for automation and consideration of ways of doing things in a more planned fashion. As the demographic dividend passes and there are few or no places to get a large labour force the pressure to innovate will return. So will the requirement to do things right the first time.

No it is not the answer.  The amount of testing we can do will always outrun the number of people who can do it.  There is always more testing that one can add to try and assure that less defects are in the system.  However, that will only take you so far.  Inserting less defects in the first place and better planning to concentrate the resources where they are required is a better solution.  Next week we will look at the demographic dividend as it relates to testing (and a lot of other industries).

If we refer to the dictionary definition of Oracle we find a couple of possible definitions that could be applied to the use made by most people of this term although a couple might lead to some questions.

One which raises a couple of questions is an ambiguous or obscure utterance – not what we want in software testing.  The second is a divine communication – very unusual in software testing.

Probably the most relevant one is the which suggests that the person who is delivering wise and potentially influential pronouncements in an orcale.    This is the definition that comes closest to the use made of this term in Software Quality Assurance and Quality Control areas.  The idea is that the person knows more and can use that knowledge to provide better Quality Control in some fashion.

The issue is that it is only as good as the person making the pronouncement.  If they are wrong, then that may lead to a cascade of mistesting and wasted time.  The second part is that it is only good as long as the person is in the place where they can claim to be oracular.  Once they leave all the information and knowledge goes out the door with them.

Quality Assurance and Process Improvement solves this problem.