NVP Software Solutions

Tag: Process Improvement

  • Quality Assurance Assessments – Part 4

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal. This week we will discuss What to do with the results of a QA Assessment now that we have completed an explanation of HOW to do Assessment in a past blog.

    What to with the results of a QA Assessment

    There is a strong temptation to (facetiously) say Do Nothing with the Results since that happens so frequently. The Assessment is completed and everyone just wants to forget about it. Not only is that a direct waste of the effort and time included in the assessment, it also sends the signal to everyone that their effort was unnecessary and their thoughts unappreciated. Don’t expect a lot of effort next time under this scenario.

    If we use the example from the last blog (referenced above) of the questionnaire or in person interviews to elicit the information using open-ended questions, then we will end up with a lot of disparate information that may not be readily parsed.

    The steps are as follows:

    1. Review all the provided answers.
    2. During the review write down some general categories for the answers (i.e. insufficient testing; requirements issues; development issues; testing issues). If these categories were predetermined then this step does not apply.
    3. Allocate the answers into the categories.
    4. Allocate the answers that fit into more than one category (put them into both).
    5. Allocate the answers that only occur once and do not fit into any category (make a category of Other and put them there).
    6. Extract a common consensus from each category (there is a lot of work in this step)
    7. Start a process of finding the root cause of the common problems.

    Now we have to act on the root causes and resolve them. This could be a whole series of blogs but we will leave that for the process improvement cycle.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. Has anyone acted on the results?
    3. Were the results used for Process Improvement?

    Next Week: Vocabulary

  • Quality Assurance Assessments – Part 3

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss HOW to do a QA Assessment now that we have completed an explanation of WHY to do Assessment in a past blog.

    HOW Quality Assurance Assessments are Done:

    The following needs to be done in order to complete a Quality Assurance Assessment:

    1. Determine the objective of the assessment. (Refer to Why Quality Assurance Assessments are Done in a past blog).
    2. Set up a team (may be a team of 1) to do the assessment.
    3. Determine the targeted group who are going to provide information to the team.
    4. Determine the method of getting the information (questionnaire; in person interviews; survey).
    5. Using the method selected above, carry out the assessment.
    6. Collate the results.
    7. Provide a report.

    The above is a general methodology. The following is a short example of a very basic QA Assessment.

      1. The objective is to determine how well the Software Testing Process worked on the last project.
      2. The team will be the Quality Assurance department (not involved in the particular project).
      3. Target audience: Software Testers, Developers, Project Manager(s), End Users, Management, all other interested stakeholders.
      4. Methodology: Individual interviews using a questionnaire. Some sample questions follow
        • What went right with the project in terms of Software Testing?
        • What went wrong with the project in terms of Software Testing?
        • What expectations were and were not met?
        • How could the process be improved?

    This is a very small sample of questions to be answered. From there:

    1. Compile the answers into a report removing the names and any identifying comments.
    2. Create a set of recommendations based on the results.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: KWSQA, TASSQ and London Peer-to-Peer

  • Quality Assurance Assessments – Part 2

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss WHY we want to do a QA assessment now that we have completed an explanation of WHAT an Assessment is in a past blog.

    WHY Quality Assurance Assessments are Done:

    1. They’re Mandated

    If it is mandated but not accepted then it is an exercise in futility. This is quite possibly the worst reason to start a QA Assessment, especially from the viewpoint of those carrying out the assessment. The process will be completed with little effort given, in the shortest time possible and the results will end up on a shelf, never to be discussed again.

    2. To Find Out What Went Wrong

    In this case, there will be an effort to find out what went wrong in the just-completed project and even if no further action is taken, at least people are aware of what everyone considered to be problems (which may not have been obvious to other people in the project). The danger is that this session and the final report can easily turn into an exercise of complaint and self justification.

    3. To Improve the Process

    This is by far the best justification for a Test Process Assessment. If parties are focused on what needs to be done to make the next effort easier, better, or more effective, then the discussion will focus on finding solutions to problems encountered. The participants cannot simply list their problems and concerns and then walk away.

    We want to finish with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: Test Plans

  • Quality Assurance Assessments – Part 1

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. No matter where each assessment lands on the spectrum, they are intended to do only one thing: Improve the Processes that are used. This week we will discuss the various forms of assessments.

    Post Project

    Post project assessments have had a variety of names throughout the years:

    1. Post-Implementation Review
    2. Lessons Learned
    3. Post Mortem

    Regardless of what they are called, they all have one purpose. The people on the project are asked to recall what went right and what went wrong in a project. This could mean trying to remember things that are a few years old, which isn’t always ideal. The person conducting the assessment then provides the information to a recorder. The resulting information is compiled somewhere and (sometimes) used for process improvement. (More on that in our next blog series).

    In-Project Assessments

    In-Project Assessments can have a variety of formats:

    1. Walkthroughs
    2. Reviews (checkpoint and in process)
    3. Inspections

    Sometimes the above are referred to as Reviews and not as Assessments.

    Test Process Assessment

    The last type of Assessment we want to discuss is the Test Process Assessment. This one is aimed solely at assessing the Test Process used for any particular project. It is often carried out by an independent group although it can be done by a Quality Assurance person or a test manager. The assessment looks for places where the process is not efficient or not effective. It looks for redundancies and repetition. Frequently the assessment starts with the Quality Assurance aspect of the project but finds itself extending back into the Development piece and forward into Implementation. It’s usually difficult to carry out an assessment just of the Quality Assurance piece when there are so many other interlocking pieces that impact Quality Assurance and Quality Control. See nvp.ca/services/assessments/ for further details.

    Next Week: Test Cases

  • Quality Assurance

    Toward the end of last year we encountered a situation where our presence in a project was there to “Assure Quality” or that “Quality was Assured” simply by us being there. Although the comment was made somewhat in jest, it showed a misunderstanding of Quality Assurance that can lead to dire consequences. Quality Assurance is not a standard although it includes standards. It is a process and a journey (and not a destination).

    Quality Assurance looks at the processes that are being used and tries to find places for improvement so that everything can be smoother. It looks for root causes of existing problems and eliminates or reduces their impact so that they no longer impact the process further along. Sometimes this is referred to as “Removing the Rocks in the Stream” so that the flow of the project is smoother. Quality Assurance also tries to anticipate other items that might impede smooth project completion and eliminate those as well. This is why it is never complete. There is likely to be improvements that will still occur and we can continue to identify those opportunitiess. Even if we can’t identify opportunities for improvement it is likely that someone in the organization can see something that can be changed. Every project is slightly different and the opportunities for improvement or to change processes are also different. If the projects are not different, then they can be turned into a standard process and there is no need for any individual improvements. That will still leave process improvement as a possibility.

    It is suspected that the initial comment at the beginning of this blog was really related to Software Testing and not Quality Assurance at all. However the misconception exists.

    Next Week: Quality Assurance Centre of Excellence

  • Testing for System Integrators – Part 5

    Last week our blog discussed the remaining answers to the questions and promised that we would look in detail at two of the answers (which are somewhat similar so we will concentrate on one only).

    There is nothing in the contract (contract is signed) and there is no intention of putting anything in the contract about Quality Assurance.
    Now you have a challenge. Clearly the process is mostly done and there is absolutely no buy-in to Quality Assurance. The next question that needs to be asked is “Why have you brought Quality Assurance in if there is no interest?”

    The key steps here are to determine your position and map out your strategy. There is any number of answers to the question of “Why have you brought Quality Assurance in if there is no interest”.

    1. The final client has belatedly required it. I.e. they have realised it is an omission from the contract and now feel it is incumbent on the System Integrator to provide this as part of the deliverables. You need to determine the final clients needs and work towards those.
    2. The solution is more complex than the System Integrator thought and now they feel a need for Quality Assurance. I.e. like the client above they have realised the value provided by Quality Assurance and now want to implement it even though they were trying to avoid it earlier. There is probably still little buy-in from most of the group. You need to look at each of the Stakeholders and determine their status vis-a-vis Quality Assurance and plan to convert them all to supporters. This is a crucial piece of your strategy in order to be successful.
    3. The System Integrator’s management is becoming nervous and wants Quality Assurance there as a check. While you have management support the team may feel they have an extra burden and possibly someone who is watching them. As in the above, you will need to look at the Stakeholders and see how to convert them to supporters. Otherwise you will get no information at all.
    4. The last possibility is they want someone to blame. This is a tricky one. No matter what you do (either proactively or reactively) they may blame you. You need to plan carefully in order to make sure that your work is recognised as contributing to the success of the project. You need to be very proactive in stating what needs to be done; why it needs to be done and the benefits accruing from having it done. And make sure that everything is documented!

      Happy New year

  • Testing for System Integrators – Part 4

    Over the next few weeks, the NVP blog will focus on Software Testing for System Integrators. From NVP’s point of view, a System Integrator is someone who brings together a number of applications (from vendors), adds some glue and ends up with a solution for the organization they are working with. This seems to agree with the Wikipedia definition fairly closely. So where does Quality Assurance come into this? One would like to think early or very early in the process but that’s not always the case.

    Last week we looked at responses to the first 4 questions. This week we are continuing with the remainder:

    1. The contract states the following specifically about Quality Assurance and everyone is in agreement
    2. The contract says nothing about Quality Assurance but it’s noted as a topic and the contract will not be finalized without this discussion
    3. The contract says nothing about Quality Assurance so far, but now that you have brought it up we will add it.
    4. There is something in the contract about Quality Assurance and we can look it up for you (contracts are signed).
    5. There is nothing in the contract (contract is signed) and there is no intention of putting anything in the contract about Quality Assurance
      Now you have a challenge. Clearly the process is mostly done and there is absolutely no Buy-in to Quality Assurance. The next question that needs to be asked is “Why have you brought Quality Assurance in if there is no interest?” The answers to this and the response will be next week’s blog
    6. We don’t know (but that is a good question)
      There is some hope here and you are in a position to influence the content and results. It may be late in the process but we can try.
    7. We don’t know (and we don’t care)
      This is a similar situation to the answer to number 5. There is a challenge for Quality Assurance and that challenge must be tackled.

    Suffice to say the items in the above list have an obvious gradation from very manageable to a real challenge in the order they are presented. If you get the first answer, you’re well on your way. If you get some of the middle answers you have some work to do, but there’s still time to make change. If you get the last few answers, you are in trouble but not defeated!

    Next Week: What to do with Number 5 and 7.

  • Testing for System Integrators – Part 3

    Over the next few weeks, the NVP blog will focus on Software Testing for System Integrators. From NVP’s point of view, a System Integrator is someone who brings together a number of applications (from vendors), adds some glue and ends up with a solution for the organization they are working with. This seems to agree with the Wikipedia definition fairly closely. So where does Quality Assurance come into this? One would like to think early or very early in the process but that’s not always the case.

    Last week we provided several possible answers to our original

    1. The contract states the following specifically about Quality Assurance and everyone is in agreement
      This means that you simply have to “bridge the gap” between what is expected from the vendors and what is promised to the final client. The only problem may be that you do not agree with the contracted items.
    2. The contract says nothing about Quality Assurance but it’s noted as a topic and the contract will not be finalized without this discussion
      This is almost the best situation. While it may be a little late in the process, the willingness to add Quality Assurance exists and people are behind it.
    3. The contract says nothing about Quality Assurance so far, but now that you have brought it up we will add it.
      The same comment as above is applicable except that there is not quite the backing we might have had earlier.
    4. There is something in the contract about Quality Assurance and we can look it up for you (contracts are signed).
      Well at least they considered it; it may not be correct or complete but it was not entirely ignored. Once you find out what is in the contract you may (or may not) have concerns to handle.
    5. There is nothing in the contract (contract is signed) and there is no intention of putting anything in the contract about Quality Assurance
    6. We don’t know (but that is a good question)
    7. We don’t know (and we don’t care)

    Suffice to say the items in the above list have an obvious gradation from very manageable to a real challenge in the order they are presented. If you get the first answer, you’re well on your way. If you get some of the middle answers you have some work to do, but there’s still time to make change. If you get the last few answers, you are in trouble but not defeated!

    Next Week: What to do with the answers (remainder).