NVP Software Solutions

Blog

  • Half the money I spend…

    Half the money I spend…

    “Half the money I spend on advertising is wasted; the trouble is, I don’t know which half.” The above quote is attributed to various people. We want to apply it to Software Quality Assurance. 

    Quality Assurance is one of the areas where people (not in QA) feel that resources or funding are wasted since they see no immediate benefit. Avoiding the process improvement and incremental changes are simply piling up technical and resource debt for the future.  Bad processes do not fix themselves and usually get worse over time.  The cumulative debt increases until it is unmanageable.

    Every problem avoided is one less to fix. When creating or improving processes and applying them to any particular situation, it is necessary to measure the improvement by taking a baseline and checking for improvement or degradation.  The effort invested in measurement will pay back with further improvements.

    Software Testing solves problems for yesterday and today.

    Quality Assurance solves your problems coming tomorrow!

  • You can only predict things after they occur

    You can only predict things after they occur

    “You can only predict things after they occur” This quote was attributed to Eugene Ionescu in the source I had.

    There are several related quotes:

    “Those who have knowledge don’t predict” and its corollary “Those who predict, don’t have knowledge.”

    Or lastly: “The best way to predict the future is to create it” (Peter Drucker).

    Maybe we could use these quotes the next time someone says how long testing is going to take.  The interesting part is when you push back and say: “How long did it take last time?” or “How much bigger (in whatever the favourite measure is) is this project compared to the last one?”.  Neither of those questions elicit much useful information or a response at all so the analogous method using internal numbers is unavailable.

    If you have no previous statistics from the organisation, there are lots of statistics available online or you can use one of the Estimation Methodologies (PERT, Planning Poker, WBS, Function Point Analysis or Test Point Analysis).  They provide a good starting point but must be adjusted for the particular situation. 

    One of the major aspects of Quality Assurance is to gather up statistics on what occurred last time and use that to predict the future. While no one can get 100% accuracy, it certainly helps to know what occurred in the past.  Adjust for the situation at hand and go forward.

    Software Testing solves problems for yesterday and today.

    Quality Assurance solves your problems coming tomorrow!

  • Don’t fix issues later; fix them now – Part 2

    Don’t fix issues later; fix them now – Part 2

    Last week we discussed the problems of putting off resolving issues and the extra costs incurred.  Due to the response received we thought we would revisit the issue with one further example and how it was solved.  Please review the post from last week, and particularly the third example (reproduced below) as an introduction to the problem.

    1. Failure to agree on defect metrics lead to inconsistent counts and hours wasted reconciling expectations between the vendor and client.

    The above, more general, problem manifested itself in a particular instance shortly after a new QA Manager showed up to take over the project.  About a week later, there a ‘crisis’ about 6 missing issues recording enhancements that needed to be costed by the software vendor.  The QA manager had not seen them at that time.  The vendor seemed to have no track of them.  However, the issues were recorded in a well-known issue tracker so they weren’t lost, just somewhere with a wrong classification.  Eventually they were found and put back into active consideration.

    Process changes solved the issue for the rest of the project:

    1. An initial classification was carried out by the client QA manager and then discussed at the Daily Issue Review meeting with the vendor to ensure that all issues were correctly classified within 24 hours of being raised.
    2. The second change was to add a view to the Issue Dashboard to list issues that had not been touched for 3 days.  This was a secondary check in case something was misassigned and was not begin addressed. It would be picked up 3 days later and checked.  N.B. Obviously this could be gamed by having someone go through and look at every issue every couple of days and stop them landing on the list.  See point 3.
    3. Every month and more frequently as the project came close to completing, a full listing of all Not Closed issues was reviewed just to ensure that nothing was being lost or simply moved forward without resolution.

    ‘Crisis’ solved and it did not return for the duration of the project. 

    Clearly, if there were few enough open issues, then memory would be enough to know what should be there.  In a multi year project with many testers and an external vendor this was more critical.

    Software Quality Assurance solves problems for yesterday, today and tomorrow.

  • Don’t fix issues later; fix them now!

    Don’t fix issues later; fix them now!

    “Don’t fix issues later; fix them now ” or “Never put off to tomorrow what you can do today”. This seems so obvious but try doing it in a project when the schedule and budget are under pressure.  If this is restricted to Software Testing then issue classification and prioritisation can help to clarify which issues are worth fixing and which can be safely left for a future release. 

    However, if the issue relates to a process used in the project, then not only does the issue potentially impact the schedule and budget at the time it is discovered, but it will impact the project each time that process is repeated.  Fixing it now will not only save the current project time and money; it will save the coming projects a lot more assuming it is documented and the fix is implemented.  Some processes we have seen add time and budget to a project include the following:

    1. Failure to classify defects and move them to the relevant group allowed one person to repeatedly reclassify defects and change priorities until the software vendor bogged down in multiple half completed code changes.
    2. Moving ahead with documentation that had not been fully reviewed and approved lead to more changes and a huge recoding effort.  In particular, adding an approval step to every piece of documentation in a system meant opening every function to code an extra step.
    3. Failure to agree on defect metrics lead to inconsistent counts and hours wasted reconciling expectations between the vendor and client.

    Fixing any of these would have saved millions of dollars and years of time on the project.

    Software Quality Assurance solves problems for yesterday, today and tomorrow.

  • Quality Throughout the Organization

    Quality Throughout the Organization

    Many articles and groups address Quality Assurance and Quality Management in isolation and do not show the linkage between the two. However, that is not the way they work.

    There is no chance that a Quality Assurance initiative or an attempt to install Quality Management in isolation will work.

    Every one of the sources on Quality Assurance (most external to Software) mention the need for the concept of Quality to permeate the entire organization to be effective.  Even with the assistance of AI, there is a need to ensure that best practices for your organization are recorded and available.  As a starting point, what are the answers to the following questions.

    1. Is information from projects being shared? This is not just software knowledge and techniques. This is every facet of the organization touched by the project or the results (usually most of it). 

    2. Is there a centralized knowledge base for best practices? Like the above comment, all the information needs to be retained. Otherwise very little information will be shared, and no-one will benefit from Lessons Learned.

    Now that we know there is a need, can we do anything about it?

    Yes: We need to let the other people know the benefit as a starting point.  Then we can Continuously Improve on known items.

  • The need for Proof in Software Testing

    The need for Proof in Software Testing

    One of the problems QA has is deciding how much proof is enough.

    Proofs are expensive and time consuming to generate. Even with new tools saving everything in video and screen shots, there is still a need to confirm that the correct items appear, and they can be found later. This is particularly true in Projects with regulatory requirements. AI can certainly help with this and reduce a lot of the manual work. But we still need to set guidelines about how long, and where the proofs are kept. They must also be secured from unauthorized access and manipulation. Any mistrusted proof will cause issues at audit time. Lastly, they must be able to be found when required.

    Proofs are also helpful when we come to testing the new version of the software. Detailed actual results can not only pinpoint unexpected and detrimental changes in the new version but they can also guide other testing that may need to be done.

    Two questions arise:

    1. Are proofs necessary – Yes.

    2. How long should they be kept – that depends on the industry and requirements but (as should be the case for all documentation) a policy for document retention needs to be written, filed, and applied.

    Either way we do not keep everything for ever.

    Proofs are a crucial part of software testing and will be called upon in the future as software testing faces scrutiny in how they tested in the event of problems in production.

  • TASSQ November Meeting is just under 2 weeks away

    TASSQ November Meeting is just under 2 weeks away

    The session will focus on the ISTQB certification and how the CSTB supports it.

    ISTQB – Vision, Syllabus Scheme, Working Groups, Partner Program

    CSTB – Goal, History, Local activities, International Activities, Volunteering

    When: Tuesday, November 25th starting at 6:15 p.m. (until 7:30 p.m.) EST
    Cost:  $10.00 CAD

    Register at https://tassq.org/events

  • Software Quality is Everyone’s Responsibility

    Software Quality is Everyone’s Responsibility

    Last month we talked about how October was Quality Month. We want to follow on from that with a comment about how “Quality is Everyone’s Responsibility”.  This came up repeatedly at the conference last week (amongst all the talks on AI in QA)

    If you have heard any of the following statements you may be in trouble.  Our response is in the second line of each listed comment.

    1. We are doing quality.
    Quality is not done. It is a process that is on going.

    2. We have a Quality Assurance department and Quality is their concern.
    Quality is not just one department.

    3. We have finished the Quality Assurance.
    Quality is never done.

    4. Quality Assurance is built into our processes and we have no need to change.
    You were perfect from the start?

    5. We have never had a Quality Assurance problem.
    Now is a good time to stop the first one.

    Quality cannot be added to a product that is poor quality from the start. It is not something that can be inserted into a product after it is built.

    If you feel that this is you, get in touch to discuss where you can improve.  www.nvp.ca/contact