NVP Software Solutions

Tag: Assessment

  • A better way to Test – 5

    A better way to Test – 5

    In our last blog we discussed how more testing or a higher percentage of test case coverage does not necessarily lead to better testing. We did not, however, provide a solution to the problem.

    One of the first items is to classify the testcases. In a project of any size, with numerous testcases, it is difficult to determine what each testcase is doing without classifying them. The Test Management tools allow for a lot of this and it is usually simply a matter of selecting the fields you want to use for classification; providing a limited list of choices for most of the fields (some can be freeform text entry but some need to have a limited number of choices so as to allow classification) and deciding where each testcase belongs. It will not be perfect the first time but can be refined over time. Some items that can be used for classification are as follows:

    1. Requirements (this comes out of Traceability).
    2. Function tested (may be a subset of the Requirements or it may be a separate mapping).
    3. Date of testcase creation (just so you know how old it is).
    4. Priority of the testcase.
    5. Some statistics generated from the testcase execution.
    6. The risk it tests – we have rarely seen this explicitly listed.

    The last item is the most important if you are testing against risk. It presupposes that you have the Risks listed, classified and numbered. Then it is a reasonably easy process to link the testcases.

    When your project is over, you can determine if the testcases addressed the risks (where appropriate) and whether they are useful for next time.

    Take a look at some of the seminars that we offer that address this situation and see if they apply to your situation. Testing can be better.
    Contact us for further information.

  • Selecting Software to run your business – 3

    Implementation is by far the most interesting and challenging part of Selecting Software to run your business. This is where the details become major problems and resistance to change can derail the entire project.

    It is crucial to do the following:

    1. Get everyone on side.
    2. Get a champion (or champions).
    3. Anticipate the issues.
    4. Address the issues (before they become problems).
    5. Start small (if possible).
    6. Demonstrate successes.
    7. Roll out to the entire company

    Many solution packages look simple on the outside but turn out to be complex to implement and maintain. Others have hidden depths you will never use while others will disappoint you. The selection process is crucial to anticipate some of these issues and make provisions to address them.

    No matter what package is selected or what deployment process is used; at some point the implementation must go ahead or else it will never occur.

    A study from some years ago provided the following figures for software that was purchased to be implemented:

    1. Software Delivered but never successfully used – 47% of the total.
    2. Software paid for but not delivered – 29% of the total.
    3. Software used but extensively reworked or later abandoned – 19% of the total.
    4. Software that could be used after changes – 3% of the total.
    5. Software that could be used as delivered – 2% of the total.

    Implementations have to fall into that 2% to be fully succesful and within the 5% to be partially sucessful. These are not high odds and to be successful requires a correct deployment process throughout the organisation.

    If you have concerns about any of the three stages (Stage 1 and Stage 2 were described in earlier blogs) of acquiring and implementing software – call us at 1-800-811-4718 or contact us.

    We are here to help.

  • Quality Assurance Assessments – Part 4

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal. This week we will discuss What to do with the results of a QA Assessment now that we have completed an explanation of HOW to do Assessment in a past blog.

    What to with the results of a QA Assessment

    There is a strong temptation to (facetiously) say Do Nothing with the Results since that happens so frequently. The Assessment is completed and everyone just wants to forget about it. Not only is that a direct waste of the effort and time included in the assessment, it also sends the signal to everyone that their effort was unnecessary and their thoughts unappreciated. Don’t expect a lot of effort next time under this scenario.

    If we use the example from the last blog (referenced above) of the questionnaire or in person interviews to elicit the information using open-ended questions, then we will end up with a lot of disparate information that may not be readily parsed.

    The steps are as follows:

    1. Review all the provided answers.
    2. During the review write down some general categories for the answers (i.e. insufficient testing; requirements issues; development issues; testing issues). If these categories were predetermined then this step does not apply.
    3. Allocate the answers into the categories.
    4. Allocate the answers that fit into more than one category (put them into both).
    5. Allocate the answers that only occur once and do not fit into any category (make a category of Other and put them there).
    6. Extract a common consensus from each category (there is a lot of work in this step)
    7. Start a process of finding the root cause of the common problems.

    Now we have to act on the root causes and resolve them. This could be a whole series of blogs but we will leave that for the process improvement cycle.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. Has anyone acted on the results?
    3. Were the results used for Process Improvement?

    Next Week: Vocabulary

  • Quality Assurance Assessments – Part 3

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss HOW to do a QA Assessment now that we have completed an explanation of WHY to do Assessment in a past blog.

    HOW Quality Assurance Assessments are Done:

    The following needs to be done in order to complete a Quality Assurance Assessment:

    1. Determine the objective of the assessment. (Refer to Why Quality Assurance Assessments are Done in a past blog).
    2. Set up a team (may be a team of 1) to do the assessment.
    3. Determine the targeted group who are going to provide information to the team.
    4. Determine the method of getting the information (questionnaire; in person interviews; survey).
    5. Using the method selected above, carry out the assessment.
    6. Collate the results.
    7. Provide a report.

    The above is a general methodology. The following is a short example of a very basic QA Assessment.

      1. The objective is to determine how well the Software Testing Process worked on the last project.
      2. The team will be the Quality Assurance department (not involved in the particular project).
      3. Target audience: Software Testers, Developers, Project Manager(s), End Users, Management, all other interested stakeholders.
      4. Methodology: Individual interviews using a questionnaire. Some sample questions follow
        • What went right with the project in terms of Software Testing?
        • What went wrong with the project in terms of Software Testing?
        • What expectations were and were not met?
        • How could the process be improved?

    This is a very small sample of questions to be answered. From there:

    1. Compile the answers into a report removing the names and any identifying comments.
    2. Create a set of recommendations based on the results.

    If you are having trouble working this out, contact us and we can help guide you and your team in the right direction.

    Finally, we’ll leave you with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: KWSQA, TASSQ and London Peer-to-Peer

  • Quality Assurance Assessments – Part 2

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. This week we will discuss WHY we want to do a QA assessment now that we have completed an explanation of WHAT an Assessment is in a past blog.

    WHY Quality Assurance Assessments are Done:

    1. They’re Mandated

    If it is mandated but not accepted then it is an exercise in futility. This is quite possibly the worst reason to start a QA Assessment, especially from the viewpoint of those carrying out the assessment. The process will be completed with little effort given, in the shortest time possible and the results will end up on a shelf, never to be discussed again.

    2. To Find Out What Went Wrong

    In this case, there will be an effort to find out what went wrong in the just-completed project and even if no further action is taken, at least people are aware of what everyone considered to be problems (which may not have been obvious to other people in the project). The danger is that this session and the final report can easily turn into an exercise of complaint and self justification.

    3. To Improve the Process

    This is by far the best justification for a Test Process Assessment. If parties are focused on what needs to be done to make the next effort easier, better, or more effective, then the discussion will focus on finding solutions to problems encountered. The participants cannot simply list their problems and concerns and then walk away.

    We want to finish with a few questions and ask you to post your answers.

    1. Have you participated in a Test Process Assessment?
    2. What was the justification for the Assessment?
    3. Were the results used for Process Improvement?

    Next Week: Test Plans

  • Quality Assurance Assessments – Part 1

    Quality Assurance Assessments take a variety of forms in an IT project and can range from very informal to very formal in nature. No matter where each assessment lands on the spectrum, they are intended to do only one thing: Improve the Processes that are used. This week we will discuss the various forms of assessments.

    Post Project

    Post project assessments have had a variety of names throughout the years:

    1. Post-Implementation Review
    2. Lessons Learned
    3. Post Mortem

    Regardless of what they are called, they all have one purpose. The people on the project are asked to recall what went right and what went wrong in a project. This could mean trying to remember things that are a few years old, which isn’t always ideal. The person conducting the assessment then provides the information to a recorder. The resulting information is compiled somewhere and (sometimes) used for process improvement. (More on that in our next blog series).

    In-Project Assessments

    In-Project Assessments can have a variety of formats:

    1. Walkthroughs
    2. Reviews (checkpoint and in process)
    3. Inspections

    Sometimes the above are referred to as Reviews and not as Assessments.

    Test Process Assessment

    The last type of Assessment we want to discuss is the Test Process Assessment. This one is aimed solely at assessing the Test Process used for any particular project. It is often carried out by an independent group although it can be done by a Quality Assurance person or a test manager. The assessment looks for places where the process is not efficient or not effective. It looks for redundancies and repetition. Frequently the assessment starts with the Quality Assurance aspect of the project but finds itself extending back into the Development piece and forward into Implementation. It’s usually difficult to carry out an assessment just of the Quality Assurance piece when there are so many other interlocking pieces that impact Quality Assurance and Quality Control. See nvp.ca/services/assessments/ for further details.

    Next Week: Test Cases