NVP Software Solutions

Tag: Quality Assurance

  • Upcoming Software Testing & Quality Assurance Events – April 2016

    NVP Software Solutions will be participating in the following three software testing and quality assurance events happening this April in Ontario, Canada. The events are located in Toronto, Kitchener-Waterloo and London in the coming two weeks. Check out the relevant websites for more information and to register. This is a great opportunity to connect with other software testing and quality assurance professionals. We hope to see you there!

     

    Toronto Association of Systems & Software Quality

    TASSQ – Toronto Association of System and Software Quality – Everything you Wanted to Know about the CSQE! – Brenda Fisk, Director,
    ASQ Canada Deputy Regional Director 2014-2016
    Software Division, Division Executive Team – April 26, 2016 – See http://www.tassq.org/

     

     

    Software Testing in Kitchener Waterloo

     

    KWSQA – Kitchener Waterloo Software Quality Association – the bare minimum you need to know about web application security in 2016 – Ken De Souza – April 27, 2016 – See www.kwsqa.org

     

     MANHATTAN

    London Quality Assurance Peer-to-Peer Contact neil@nvp.ca for more details

  • Quality Assurance Assessments – Part 3

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss HOW to do a QA Assessment now that we have completed an explanation of WHY to do Assessment in a past blog.

    HOW Quality Assurance Assessments are Done:

    The following needs to be done in order to complete a Quality Assurance Assessment:

    1. Determine the objective of the assessment. (Refer to Why Quality Assurance Assessments are Done in a past blog).
    2. Set up a team (may be a team of 1) to do the assessment.
    3. Determine the targeted group who are going to provide information to the team.
    4. Determine the method of getting the information (questionnaire; in person interviews; survey).
    5. Using the method selected above, carry out the assessment.
    6. Collate the results.
    7. Provide a report.

    The above is a general methodology. The following is a short example of a very basic QA Assessment.

      1. The objective is to determine how well the Software Testing Process worked on the last project.
      2. The team will be the Quality Assurance department (not involved in the particular project).
      3. Target audience: Software Testers, Developers, Project Manager(s), End Users, Management, all other interested stakeholders.
      4. Methodology: Individual interviews using a questionnaire. Some sample questions follow
        • What went right with the project in terms of Software Testing?
        • What went wrong with the project in terms of Software Testing?
        • What expectations were and were not met?
        • How could the process be improved?

    This is a very small sample of questions to be answered. From there:

    1. Compile the answers into a report removing the names and any identifying comments.
    2. Create a set of recommendations based on the results.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: KWSQA, TASSQ and London Peer-to-Peer

  • Software Testing Strategy – The Test Plan

    Any given test plan and its contents vary widely depending on to who is doing the testing and what template is used. Test plans can be found anywhere on the web. They can be generic in nature or specific to a particular industry and have specific sections that are critical for regulatory or risk reasons. A test plan can also be seen as a strategy, while in some cases strategy is never considered. Sometimes test cases are included in a test plan.

    We consider a test plan to be a document that outlines a path for the testing but excludes the test cases (which we like to keep in a database). The test plan often includes the following (not a comprehensive list):

    1. Introduction
    2. Test Approach
    3. Assumptions and Dependencies
    4. Risks and the Risk Plan
    5. Schedule and Resources
    6. Glossary

    The Incredible Shrinking Test Plan

    We’ve seen where one client invested a lot of time creating a test plan template for use within their organization. The template with various sections indicated what the contents of that section should include, however, if the section was irrelevant,  a statement as to why it was inapplicable was to be noted. Under no condition was any section to be deleted. Unfortunately, one group misunderstood the importance of keeping ALL data and deleted any section they felt was unnecessary. They innacurately used the previous plan version as the template for the next one and so on. So every time a section was deleted it was lost forever. We used to refer to those plans as the The Incredible Shrinking Test Plan, it just got smaller and smaller.

      1. Do you write test plans?
      2. If you use a template, do you have trouble filling it out?
      3. Do you get test plans reviewed by the relevant stakeholders?
      4. Are they approved at any point?
      5. What is your experience in how well they test what you need to test?

    And most importantly: Do you ever review them after the project is over and see how well you adhered to the initial outline?

    Next Week: Assessments – How

  • Quality Assurance Centre of Excellence – Part 6

    Over the past 4 weeks we have been discussing the following list of items related to Quality Assurance. While it may not necessarily be a full comprehensive list, it is a starting point for many Quality Assurance activities. In looking at the list, there are many things a tester could concentrate on. We also have to be concerned about perceptions of the Quality Assurance department. Concentrating on any one of the items to the exclusion of the rest can not only cause one to lose sight of the larger picture, it can also cause problems with the remainder of the organization that feels that Quality Assurance has one one agenda.

    1. Proactive
    2. Analytical
    3. Focused on Root Causes
    4. Focused on Process

    Given the above list, what can a Quality Assurance Centre of Excellence concentrate on in order to provide value to the organization?

    Proactive We need to spend some time each week looking at the future and how Quality Assurance can address it. The first few weeks are difficult because you may not know what to look at or look for and you may not be aware of what to ask. Some initial interviews asking for expectations might help guide the agenda. The answers need to be analysed and weighed so that emphasis can be placed on the correct items. Be prepared for your initial ideas to be wrong and for them to change over time.

    Analytical This one can be more political. People do not like having their work focused on with the intent of seeing what has gone wrong. You will need to repeat the words ‘We are here to help’ constantly, and demonstrate that you are living up to them. Since this is reasonably concrete, it is easy to spend all your time here and start generating reports that may or may not be useful. Results from this item need to be reviewed to check for their usefulness.

    Focused on Root Causes We already indicated what this one was and what would be involved. It is easy to spend a lot of time on this and not get anywhere. After all, if Root Causes were easy to find, they would already have been discovered and corrected. Brainstorming on the Root Cause followed by a time for reflection is probably best. Be prepared to be surprised as to what the Root Cause really is followed by resistance to any sort of correction.

    Focused on Process There are entire books and courses on Process Improvement. Much of the above will drive the processes since it is the processes that are wrong and are causing the problems in the first place. So while you can do Process Improvement in isolation, it can also be driven by what comes from the previous three sections. Be prepared for some long payback cycles and a lot of resistance.

    Visit our website for ways we can help your organization.

    Next week Assessments

  • Quality Assurance Centre of Excellence – Part 5

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the root cause analysis aspect of a QA CofE. This week we want to focus on the process

    Focused on Process – This also applies to all of our previous points and they can all be applied at any level. The Quality Assurance CoE could look at very general processes for the entire organization or very specific processes to resolve a specific problem.

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Process:

    1. Analyzing a process can be more difficult than analyzing a product
    2. The payback may be longer term

    If a process is not followed correctly then any measurements taken of the process may be suspect. In addition, if no measurements have been defined before the process was implemented, then it is difficult to retroactively add them to the process.

    There are a couple of solutions to the above issues:

    • Build in compliance via automated enforcement. In other words, the compliance to the process is forced by checks built in to the process and enforced automatically. Many computer systems do a lot of automated enforcement already. For a process, we just need to abstract that to the process level and see how it can be made self-enforcing
    • The measurements need to be correctly defined (not a trivial problem) and then added to the process so they are automatically generated as a result of doing the process.
    • Finally as a sequel to the automated generation of the measurements, the measurement of the expected benefit needs to be well defined and implemented before the process is used. We probably want to ignore the first few measurements as trials and not used for actual decisions. Once the process has settled down, we can start measuring and collating the statistics.

    Next week QA CoE Summary

  • Quality Assurance Centre of Excellence – Part 4

    Our current blog series focuses on the Quality Assurance Centre of Excellence. Last week we covered the analytics aspect of a QA CofE. This week we want to focus on the root cause analysis.

    Focused on Root Causes – This can be applied to both of the previous blogs. When it comes to being proactive, the Quality Assurance CoE could look at Root Causes of potential issues and plan around them or else they could look at Root Causes from their analysis and work to understand and solve them. Regardless of which way this is dealt with, the problem could be solved at different levels. We could be looking at Root Causes for software development and deployment or we could be looking at root causes that impact the entire organization (structure, culture, management).

    There are two challenges that a Quality Assurance Specialist usually faces with respect to Root Cause Analysis:

    1. The Root Cause may be difficult to find.
    2. Once found, there may be resistance to fixing the Root Cause

    An example of the first occurs when someone is supplying information to another group and in the process of supplying it the information is manipulated or reformatted in some way. The manipulation adds in an error which is not obvious. The next group uses the erroneous data to make decisions. The error is not large enough to cause major problems but it has an impact further along. No one notices the problem until much later. This makes the Root Cause difficult to find and difficult to correct since no one is recognizing the initial error.

    The second is a much worse problem. The Root Cause has been identified and the solution created. Then there is resistance to making the required changes:

    • The problem may be outside your department.
    • Someone may have to admit they are wrong and have been wrong for years.
    • There may be costs attached to making the change and they will fall on someone else.

    All of the above can lead to resistance to the change. Next week’s blog will address these.
    Next week Process Focus

  • Quality Assurance Centre of Excellence – Part 3

    This blog series will on the Quality Assurance Centre of Excellence. Last week we covered the proactive aspect of a QA CofE. This week we want to focus on the analytic component.

    Analytics – The analytical aspect of quality assurance looks at data and analyzes what went right and what went wrong. Information comes from statistics and Post Implementation Reviews. The Quality Assurance Centre of Excellence gathers the data, collates it, and then provides information based on that data. They may act on the data, or they may simply pass the information along assuming that someone else will act on it within a specific project.

    Gathering analytics is probably the most traditional role for a Quality Assurance Specialist. They gather statistics from current processes and analyze them to determine what is working well, what can be used elsewhere, and what needs to be fixed.

    There are two challenges that the Quality Assurance Specialist usually faces:

    1. Data is gathered because it always has been and information continues being gathered whether or not there is any current use or anyone even viewing it.
    2. The Quality Assurance analyst carries out all sorts of detailed analysis and then nothing is done with the results. This can be very discouraging and isn’t a great use of time and work efforts.

    There are ways of addressing the above two above challenge:

    First, there should be a yearly review of all the statistics gathered with insight given as to if, how and where they are being used. This is not a review of the value of the information, it is simply a statement of whether they are being used somewhere in the organization. More extensive reviews might ask the deeper question of the value of any use of the results.

    The second challenge above is a little harder to deal with. Part of it can be addressed by ensuring that any new requests for results do have value (being Proactive) and then look at why the existing items are not being used. If they do not demonstrate some value to the organization, then they are not going to be used. Again, we want to ensure the initial request for the analysis has value attached; ongoing value is being demonstrated, and the yearly review incorporates this component to make sure it is still of value.

    Now that we have all the information, we can start using it proactively to help make our current work and initiatives more efficient.

    Next week Root Cause Focus

  • Quality Assurance Centre of Excellence – Part 2

    This blog series will be focusing on the Quality Assurance Centre of Excellence. Last week we covered the different concentrations of the QA CofE. This week we want to specifically focus on the proactive component.

    Be Proactive – In this scenario the Quality Assurance CoE would be looking ahead and seeing what can be anticipated. They could be looking at coming projects; coming changes in organization direction; new technologies that might impact the organization; and the entire environment in which the organization operates in an effort to anticipate the impact on technology. Big data and security come immediately to mind for many organizations. There is almost no end to what could be considered.

    So where do we start? The answer is we start with what we are currently doing in terms of projects, technology, and reacting to the environment. This is sometimes called Setting a Baseline although we may need something more general than that. We may also need to go into the past a little to get understand overall trends. While some things can change  overnight, a lot of the major trends are obvious many years in advance; just not recognized at the time. We can use the quote from George Santayana here. Once we know where we have been and where we are now, then it is time to look at the competition to see where they may be ahead of us or where they appear to be going.

    Now that we have all the information, we can start anticipating what might impact our organization.

    1. Is the environment going to change? If it is a government driven change, it is rarely a surprise and can be anticipated. We start to position ourselves to be ready for the change so it is a non-event for us when it occurs.
    2. Is technology going to change? YES – that’s inevitable. But, what impact will it have on my particular business and how fast will it happen are the more important, harder-to-answer questions. We need to look at our customers (How quickly do they start using new technology?); our own organization (What is the history of technology adoption?) and our competitors (Are they using technology to get ahead of us). We start to position our development and Quality Assurance groups to be ready for this new technology.
    3. What are the coming projects? This means that we need to be involved in the planning processes at the highest level. Only in that way can we anticipate what might be coming. We schedule incremental changes to ensure that the departments can tackle the new projects with a minimum of disruption

    As was said: “It is difficult to make predictions, especially about the future.”

    It is recommended that you put this on your daily reminder list: What is the potential impact of today’s discussions?

    Next week Being Analytic