NVP Software Solutions

Tag: QA

  • Quality Assurance Assessments – Part 4

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal. This week we will discuss What to do with the results of a QA Assessment now that we have completed an explanation of HOW to do Assessment in a past blog.

    What to with the results of a QA Assessment

    There is a strong temptation to (facetiously) say Do Nothing with the Results since that happens so frequently. The Assessment is completed and everyone just wants to forget about it. Not only is that a direct waste of the effort and time included in the assessment, it also sends the signal to everyone that their effort was unnecessary and their thoughts unappreciated. Don’t expect a lot of effort next time under this scenario.

    If we use the example from the last blog (referenced above) of the questionnaire or in person interviews to elicit the information using open-ended questions, then we will end up with a lot of disparate information that may not be readily parsed.

    The steps are as follows:

    1. Review all the provided answers.
    2. During the review write down some general categories for the answers (i.e. insufficient testing; requirements issues; development issues; testing issues). If these categories were predetermined then this step does not apply.
    3. Allocate the answers into the categories.
    4. Allocate the answers that fit into more than one category (put them into both).
    5. Allocate the answers that only occur once and do not fit into any category (make a category of Other and put them there).
    6. Extract a common consensus from each category (there is a lot of work in this step)
    7. Start a process of finding the root cause of the common problems.

    Now we have to act on the root causes and resolve them. This could be a whole series of blogs but we will leave that for the process improvement cycle.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. Has anyone acted on the results?
    3. Were the results used for Process Improvement?

    Next Week: Vocabulary

  • Software Testing Strategy – The Test Strategy

     

    We discussed Test Plans and their contents in the earlier blog and today we will discuss the Test Strategy. Some people will look at the title above and think we are just repeating our blog from three weeks ago. Luckily, that’s not the case. We differentiate between Test Plans and Test Strategies.

    NVP considers a Test Strategy to be a document that outlines a long term direction for testing. It’s possible that the Test Plans for each project will borrow heavily from the Test Strategy and may even be based on it. We’re not necessarily considering that level of connection in this scenario.

    The following is a set of contents for a Test Strategy

    1. Introduction – outlines the purpose of the document
    2. Corporate Business Strategy – Copy and paste from the relevant business document
    3. A high level outline of the Test Strategy – See below
    4. Alignment with corporate strategy and identification of the gaps
    5. Baseline of current testing (where we are today)
    6. Gaps between the baseline and Test Strategy
    7. Goals or methodologies to close the gaps
    8. Strategy assessment process
    9. Continuous improvement

    The Test Strategy section would include answers to the following:

    1. Where is the best place to complete testing in the life cycle?
    2. Who are the best resources for each level and type of testing?
    3. What is the long term development process for the identified resources?
    4. From where are the resources going to be sourced?
    5. What is the long term automation process or direction?
    6. Where is the greatest ROI for testing?

    Some questions to assess your Test Strategy readiness:

    1. Do you have a Test Strategy?
    2. If you have a Test Strategy; has it been reviewed recently?
    3. Is there a person or a group responsible for the Test Strategy?

    If you answered No to any of the above questions, send us an email to see if a Test Strategy would be a fit for your organization.

    Next Week: Assessments – Deal with Results

  • Upcoming Software Testing & Quality Assurance Events – April 2016

    NVP Software Solutions will be participating in the following three software testing and quality assurance events happening this April in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there!

     

    Toronto Association of Systems & Software Quality

    TASSQ – Toronto Association of System and Software Quality – Everything you Wanted to Know about the CSQE! – Brenda Fisk, Director,
    ASQ Canada Deputy Regional Director 2014-2016
    Software Division, Division Executive Team – April 26, 2016 – See http://www.tassq.org/

     

     

    Software Testing in Kitchener Waterloo

     

    KWSQA – Kitchener Waterloo Software Quality Association – the bare minimum you need to know about web application security in 2016 – Ken De Souza – April 27, 2016 – See www.kwsqa.org

     

     MANHATTAN

    London Quality Assurance Peer-to-Peer Contact neil@nvp.ca for more details

  • Quality Assurance Assessments – Part 3

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss HOW to do a QA Assessment now that we have completed an explanation of WHY to do Assessment in a past blog.

    HOW Quality Assurance Assessments are Done:

    The following needs to be done in order to complete a Quality Assurance Assessment:

    1. Determine the objective of the assessment. (Refer to Why Quality Assurance Assessments are Done in a past blog).
    2. Set up a team (may be a team of 1) to do the assessment.
    3. Determine the targeted group who are going to provide information to the team.
    4. Determine the method of getting the information (questionnaire; in person interviews; survey).
    5. Using the method selected above, carry out the assessment.
    6. Collate the results.
    7. Provide a report.

    The above is a general methodology. The following is a short example of a very basic QA Assessment.

      1. The objective is to determine how well the Software Testing Process worked on the last project.
      2. The team will be the Quality Assurance department (not involved in the particular project).
      3. Target audience: Software Testers, Developers, Project Manager(s), End Users, Management, all other interested stakeholders.
      4. Methodology: Individual interviews using a questionnaire. Some sample questions follow
        • What went right with the project in terms of Software Testing?
        • What went wrong with the project in terms of Software Testing?
        • What expectations were and were not met?
        • How could the process be improved?

    This is a very small sample of questions to be answered. From there:

    1. Compile the answers into a report removing the names and any identifying comments.
    2. Create a set of recommendations based on the results.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: KWSQA, TASSQ and London Peer-to-Peer

  • Quality Assurance Centre of Excellence – Part 6

    Over the past 4 weeks we have been discussing the following list of items related to Quality Assurance. While it may not necessarily be a full comprehensive list, it is a starting point for many Quality Assurance activities. In looking at the list, there are many things a tester could concentrate on. We also have to be concerned about perceptions of the Quality Assurance department. Concentrating on any one of the items to the exclusion of the rest can not only cause one to lose sight of the larger picture, it can also cause problems with the remainder of the organization that feels that Quality Assurance has one one agenda.

    1. Proactive
    2. Analytical
    3. Focused on Root Causes
    4. Focused on Process

    Given the above list, what can a Quality Assurance Centre of Excellence concentrate on in order to provide value to the organization?

    Proactive We need to spend some time each week looking at the future and how Quality Assurance can address it. The first few weeks are difficult because you may not know what to look at or look for and you may not be aware of what to ask. Some initial interviews asking for expectations might help guide the agenda. The answers need to be analysed and weighed so that emphasis can be placed on the correct items. Be prepared for your initial ideas to be wrong and for them to change over time.

    Analytical This one can be more political. People do not like having their work focused on with the intent of seeing what has gone wrong. You will need to repeat the words ‘We are here to help’ constantly, and demonstrate that you are living up to them. Since this is reasonably concrete, it is easy to spend all your time here and start generating reports that may or may not be useful. Results from this item need to be reviewed to check for their usefulness.

    Focused on Root Causes We already indicated what this one was and what would be involved. It is easy to spend a lot of time on this and not get anywhere. After all, if Root Causes were easy to find, they would already have been discovered and corrected. Brainstorming on the Root Cause followed by a time for reflection is probably best. Be prepared to be surprised as to what the Root Cause really is followed by resistance to any sort of correction.

    Focused on Process There are entire books and courses on Process Improvement. Much of the above will drive the processes since it is the processes that are wrong and are causing the problems in the first place. So while you can do Process Improvement in isolation, it can also be driven by what comes from the previous three sections. Be prepared for some long payback cycles and a lot of resistance.

    Visit our website for ways we can help your organization.

    Next week Assessments

  • Quality Assurance Centre of Excellence – Part 5

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the root cause analysis aspect of a QA CofE. This week we want to focus on the process

    Focused on Process – This also applies to all of our previous points and they can all be applied at any level. The Quality Assurance CoE could look at very general processes for the entire organization or very specific processes to resolve a specific problem.

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Process:

    1. Analyzing a process can be more difficult than analyzing a product
    2. The payback may be longer term

    If a process is not followed correctly then any measurements taken of the process may be suspect. In addition, if no measurements have been defined before the process was implemented, then it is difficult to retroactively add them to the process.

    There are a couple of solutions to the above issues:

    • Build in compliance via automated enforcement. In other words, the compliance to the process is forced by checks built in to the process and enforced automatically. Many computer systems do a lot of automated enforcement already. For a process, we just need to abstract that to the process level and see how it can be made self-enforcing
    • The measurements need to be correctly defined (not a trivial problem) and then added to the process so they are automatically generated as a result of doing the process.
    • Finally as a sequel to the automated generation of the measurements, the measurement of the expected benefit needs to be well defined and implemented before the process is used. We probably want to ignore the first few measurements as trials and not used for actual decisions. Once the process has settled down, we can start measuring and collating the statistics.

    Next week QA CoE Summary

  • Quality Assurance Centre of Excellence – Part 4

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the analytics aspect of a QA CofE. This week we want to focus on the root cause analysis.

    Focused on Root Causes – This can be applied to both of the previous blogs. When it comes to being proactive, the Quality Assurance CoE could look at Root Causes of potential issues and plan around them or else they could look at Root Causes from their analysis and work to understand and solve them. Regardless of which way this is dealt with, the problem could be solved at different levels. We could be looking at Root Causes for software development and deployment or we could be looking at root causes that impact the entire organization (structure, culture, management).

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Root Cause Analysis:

    1. The Root Cause may be difficult to find.
    2. Once found, there may be resistance to fixing the Root Cause

    An example of the first occurs when someone is supplying information to another group and in the process of supplying it the information is manipulated or reformatted in some way. The manipulation adds in an error which is not obvious. The next group uses the erroneous data to make decisions. The error is not large enough to cause major problems but it has an impact further along. No one notices the problem until much later. This makes the Root Cause difficult to find and difficult to correct since no one is recognizing the initial error.

    The second is a much worse problem. The Root Cause has been identified and the solution created. Then there is resistance to making the required changes:

    • The problem may be outside your department.
    • Someone may have to admit they are wrong and have been wrong for years.
    • There may be costs attached to making the change and they will fall on someone else.

    All of the above can lead to resistance to the change. Next week’s blog will address these.
    Next week Process Focus

  • Quality Assurance Centre of Excellence – Part 3

    This blog series will on the Quality Assurance Centre of Excellence. Last week we covered the proactive aspect of a QA CofE. This week we want to focus on the analytic component.

    Analytics – The analytical aspect of quality assurance looks at data and analyzes what went right and what went wrong. Information comes from statistics and Post Implementation Reviews. The Quality Assurance Centre of Excellence gathers the data, collates it, and then provides information based on that data. They may act on the data, or they may simply pass the information along assuming that someone else will act on it within a specific project.

    Gathering analytics is probably the most traditional role for a Quality Assurance Specialist. They gather statistics from current processes and analyze them to determine what is working well, what can be used elsewhere, and what needs to be fixed.

    There are two challenges that the Quality Assurance Specialist usually faces:

    1. Data is gathered because it always has been and information continues being gathered whether or not there is any current use or anyone even viewing it.
    2. The Quality Assurance analyst carries out all sorts of detailed analysis and then nothing is done with the results. This can be very discouraging and isn’t a great use of time and work efforts.

    There are ways of addressing the above two above challenge:

    First, there should be a yearly review of all the statistics gathered with insight given as to if, how and where they are being used. This is not a review of the value of the information, it is simply a statement of whether they are being used somewhere in the organization. More extensive reviews might ask the deeper question of the value of any use of the results.

    The second challenge above is a little harder to deal with. Part of it can be addressed by ensuring that any new requests for results do have value (being Proactive) and then look at why the existing items are not being used. If they do not demonstrate some value to the organization, then they are not going to be used. Again, we want to ensure the initial request for the analysis has value attached; ongoing value is being demonstrated, and the yearly review incorporates this component to make sure it is still of value.

    Now that we have all the information, we can start using it proactively to help make our current work and initiatives more efficient.

    Next week Root Cause Focus