NVP Software Solutions

Author: Neil

  • Quality Assurance Assessments – Part 2

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss WHY we want to do a QA assessment now that we have completed an explanation of WHAT an Assessment is in a past blog.

    WHY Quality Assurance Assessments are Done:

    1. They’re Mandated

    If it is mandated but not accepted then it is an exercise in futility. This is quite possibly the worst reason to start a QA Assessment, especially from the viewpoint of those carrying out the assessment. The process will be completed with little effort given, in the shortest time possible and the results will end up on a shelf, never to be discussed again.

    2. To Find Out What Went Wrong

    In this case, there will be an effort to find out what went wrong in the just-completed project and even if no further action is taken, at least people are aware of what everyone considered to be problems (which may not have been obvious to other people in the project). The danger is that this session and the final report can easily turn into an exercise of complaint and self justification.

    3. To Improve the Process

    This is by far the best justification for a Test Process Assessment. If parties are focused on what needs to be done to make the next effort easier, better, or more effective, then the discussion will focus on finding solutions to problems encountered. The participants cannot simply list their problems and concerns and then walk away.

    We want to finish with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: Test Plans

  • Upcoming Software Testing & Quality Assurance Events

    NVP Software Solutions will be participating in the following three software testing and quality assurance events happening this March in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there!

     

    Toronto Association of Systems & Software Quality

    TASSQ – Toronto Association of System and Software Quality – Building Great Apps with Real-World Testing and Automation – Mark Holland, VP of Enterprise Solutions – Applause – March 29, 2016 – See http://www.tassq.org/

     

     

    Software Testing in Kitchener Waterloo

     

    KWSQA – Kitchener Waterloo Software Quality Association – Subject still to be determined – Martin Hynie – March 30, 2016 See www.kwsqa.org

     MANHATTAN

    London Quality Assurance Peer-to-Peer Contact neil@nvp.ca for more details

  • Test Cases

    Test Cases, Test Scripts, Test Runs, Test Sets, Test Scenarios, Test Databases, Test Plans and Test Strategies have all been used to define some type of Test Case. When it comes to popular automated test tools, some of these terms do have very specific meanings. However, others are defined within testing courses or for specifically for test certifications. The problem is that many of these definitions differ. Sometimes the difference is directly in the definition; other times it is in the use. The loose terminology can lead to problems when people are expecting something different than the test case they asked for.

    For the sake of this post, we are going to define a test case as something that needs to be tested. It includes the following fields (at a minimum):

    1. Test Title
    2. Test Objective
    3. Priority
    4. Steps (or Actions)
    5. Expected Results

    Obviously this is a fairly minimal list and there are many more useful fields that could be included. We often encounter Test Cases (and there’s a lot) where there are more fields included especially if they are being recorded in one of the popular test administration tools.

    Whatever is included, we use a test case as a basis for what needs to be tested for any particular application under test. How we actually run that test case and the amount of detail recorded in the test case and about the run will differ from place to place depending on the criticality of the application and the time we have.

    We want to finish with a few questions and ask you to post your answers.

    1. Do you write test cases?
    2. Do you get them reviewed by the relevant stakeholders?
    3. Are they approved at any point?
    4. What is your experience in how well they test what you need to test?

    Next Week: KWSQA, TASSQ and London Peer-to-Peer.

  • Quality Assurance Assessments – Part 1

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. No matter where each assessment lands on the spectrum, they are intended to do only one thing: Improve the Processes that are used. This week we will discuss the various forms of assessments.

    Post Project

    Post project assessments have had a variety of names throughout the years:

    1. Post-Implementation Review
    2. Lessons Learned
    3. Post Mortem

    Regardless of what they are called, they all have one purpose. The people on the project are asked to recall what went right and what went wrong in a project. This could mean trying to remember things that are a few years old, which isn’t always ideal. The person conducting the assessment then provides the information to a recorder. The resulting information is compiled somewhere and (sometimes) used for process improvement. (More on that in our next blog series).

    In-Project Assessments

    In-Project Assessments can have a variety of formats:

    1. Walkthroughs
    2. Reviews (checkpoint and in process)
    3. Inspections

    Sometimes the above are referred to as Reviews and not as Assessments.

    Test Process Assessment

    The last type of Assessment we want to discuss is the Test Process Assessment. This one is aimed solely at assessing the Test Process used for any particular project. It is often carried out by an independent group although it can be done by a Quality Assurance person or a test manager. The assessment looks for places where the process is not efficient or not effective. It looks for redundancies and repetition. Frequently the assessment starts with the Quality Assurance aspect of the project but finds itself extending back into the Development piece and forward into Implementation. It’s usually difficult to carry out an assessment just of the Quality Assurance piece when there are so many other interlocking pieces that impact Quality Assurance and Quality Control. See nvp.ca/services/assessments/ for further details.

    Next Week: Test Cases

  • Quality Assurance Centre of Excellence – Part 6

    Over the past 4 weeks we have been discussing the following list of items related to Quality Assurance. While it may not necessarily be a full comprehensive list, it is a starting point for many Quality Assurance activities. In looking at the list, there are many things a tester could concentrate on. We also have to be concerned about perceptions of the Quality Assurance department. Concentrating on any one of the items to the exclusion of the rest can not only cause one to lose sight of the larger picture, it can also cause problems with the remainder of the organization that feels that Quality Assurance has one one agenda.

    1. Proactive
    2. Analytical
    3. Focused on Root Causes
    4. Focused on Process

    Given the above list, what can a Quality Assurance Centre of Excellence concentrate on in order to provide value to the organization?

    Proactive We need to spend some time each week looking at the future and how Quality Assurance can address it. The first few weeks are difficult because you may not know what to look at or look for and you may not be aware of what to ask. Some initial interviews asking for expectations might help guide the agenda. The answers need to be analysed and weighed so that emphasis can be placed on the correct items. Be prepared for your initial ideas to be wrong and for them to change over time.

    Analytical This one can be more political. People do not like having their work focused on with the intent of seeing what has gone wrong. You will need to repeat the words ‘We are here to help’ constantly, and demonstrate that you are living up to them. Since this is reasonably concrete, it is easy to spend all your time here and start generating reports that may or may not be useful. Results from this item need to be reviewed to check for their usefulness.

    Focused on Root Causes We already indicated what this one was and what would be involved. It is easy to spend a lot of time on this and not get anywhere. After all, if Root Causes were easy to find, they would already have been discovered and corrected. Brainstorming on the Root Cause followed by a time for reflection is probably best. Be prepared to be surprised as to what the Root Cause really is followed by resistance to any sort of correction.

    Focused on Process There are entire books and courses on Process Improvement. Much of the above will drive the processes since it is the processes that are wrong and are causing the problems in the first place. So while you can do Process Improvement in isolation, it can also be driven by what comes from the previous three sections. Be prepared for some long payback cycles and a lot of resistance.

    Visit our website for ways we can help your organization.

    Next week Assessments

  • Quality Assurance Centre of Excellence – Part 5

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the root cause analysis aspect of a QA CofE. This week we want to focus on the process

    Focused on Process – This also applies to all of our previous points and they can all be applied at any level. The Quality Assurance CoE could look at very general processes for the entire organization or very specific processes to resolve a specific problem.

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Process:

    1. Analyzing a process can be more difficult than analyzing a product
    2. The payback may be longer term

    If a process is not followed correctly then any measurements taken of the process may be suspect. In addition, if no measurements have been defined before the process was implemented, then it is difficult to retroactively add them to the process.

    There are a couple of solutions to the above issues:

    • Build in compliance via automated enforcement. In other words, the compliance to the process is forced by checks built in to the process and enforced automatically. Many computer systems do a lot of automated enforcement already. For a process, we just need to abstract that to the process level and see how it can be made self-enforcing
    • The measurements need to be correctly defined (not a trivial problem) and then added to the process so they are automatically generated as a result of doing the process.
    • Finally as a sequel to the automated generation of the measurements, the measurement of the expected benefit needs to be well defined and implemented before the process is used. We probably want to ignore the first few measurements as trials and not used for actual decisions. Once the process has settled down, we can start measuring and collating the statistics.

    Next week QA CoE Summary

  • Quality Assurance Centre of Excellence – Part 4

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the analytics aspect of a QA CofE. This week we want to focus on the root cause analysis.

    Focused on Root Causes – This can be applied to both of the previous blogs. When it comes to being proactive, the Quality Assurance CoE could look at Root Causes of potential issues and plan around them or else they could look at Root Causes from their analysis and work to understand and solve them. Regardless of which way this is dealt with, the problem could be solved at different levels. We could be looking at Root Causes for software development and deployment or we could be looking at root causes that impact the entire organization (structure, culture, management).

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Root Cause Analysis:

    1. The Root Cause may be difficult to find.
    2. Once found, there may be resistance to fixing the Root Cause

    An example of the first occurs when someone is supplying information to another group and in the process of supplying it the information is manipulated or reformatted in some way. The manipulation adds in an error which is not obvious. The next group uses the erroneous data to make decisions. The error is not large enough to cause major problems but it has an impact further along. No one notices the problem until much later. This makes the Root Cause difficult to find and difficult to correct since no one is recognizing the initial error.

    The second is a much worse problem. The Root Cause has been identified and the solution created. Then there is resistance to making the required changes:

    • The problem may be outside your department.
    • Someone may have to admit they are wrong and have been wrong for years.
    • There may be costs attached to making the change and they will fall on someone else.

    All of the above can lead to resistance to the change. Next week’s blog will address these.
    Next week Process Focus

  • Quality Assurance Centre of Excellence – Part 3

    This blog series will on the Quality Assurance Centre of Excellence. Last week we covered the proactive aspect of a QA CofE. This week we want to focus on the analytic component.

    Analytics – The analytical aspect of quality assurance looks at data and analyzes what went right and what went wrong. Information comes from statistics and Post Implementation Reviews. The Quality Assurance Centre of Excellence gathers the data, collates it, and then provides information based on that data. They may act on the data, or they may simply pass the information along assuming that someone else will act on it within a specific project.

    Gathering analytics is probably the most traditional role for a Quality Assurance Specialist. They gather statistics from current processes and analyze them to determine what is working well, what can be used elsewhere, and what needs to be fixed.

    There are two challenges that the Quality Assurance Specialist usually faces:

    1. Data is gathered because it always has been and information continues being gathered whether or not there is any current use or anyone even viewing it.
    2. The Quality Assurance analyst carries out all sorts of detailed analysis and then nothing is done with the results. This can be very discouraging and isn’t a great use of time and work efforts.

    There are ways of addressing the above two above challenge:

    First, there should be a yearly review of all the statistics gathered with insight given as to if, how and where they are being used. This is not a review of the value of the information, it is simply a statement of whether they are being used somewhere in the organization. More extensive reviews might ask the deeper question of the value of any use of the results.

    The second challenge above is a little harder to deal with. Part of it can be addressed by ensuring that any new requests for results do have value (being Proactive) and then look at why the existing items are not being used. If they do not demonstrate some value to the organization, then they are not going to be used. Again, we want to ensure the initial request for the analysis has value attached; ongoing value is being demonstrated, and the yearly review incorporates this component to make sure it is still of value.

    Now that we have all the information, we can start using it proactively to help make our current work and initiatives more efficient.

    Next week Root Cause Focus