NVP Software Solutions

Tag: QA

  • Examples of Process Improvement

    Examples of Process Improvement are sometimes a little harder to find and measure than Product Improvement. As long as you have a standard to compare a product, any change can be determined and it is usually easier to determine if the quality of the product has improved or not. Also, we may not have to actually determine the measurement methodology prior to the product being produced. We can take an existing product, determine a standard and decide if it has met that standard. Future products can then be compared against that standard. A process, on the other hand, does not necessarily have an end product. It is part of what produces the end product and may not ‘exist in the literal sense’ after it is finished.

    There are two ways we can measure Process Improvement and then determine if the Process has improved.

    • Measure the created product. This becomes product measurement and that can be used as feedback to improve the process.
    • Measure the actual process.

    The first has already been covered so we will discuss the second.
    If we make a change to the process, the fundamental question is has the process improved.
    Examples:

    • The process continues to generate product after a change with no reduction in quality. Process Improvement.
    • The process generates the same product while using less materials. Process Improvement.
    • The process generates the same product with less waste. Process Improvement.
    • The process is halted or interrupted less often than before. Process Improvement.
    • Less resources are required for the same quality product. Process Improvement.
    • The process has reduced variation. Process Improvement.

    All of the above require a proper measurement process (another process) to be in place before the process is launched. It is very difficult to measure some of the above process improvements without having an existing measurement process in place. This may eventually remind you of The Siphonaptera!

    However, if you want to find out about your own processes, take a look at our Assessment process which identifies them.

  • Process Improvement

    Ongoing Process Improvement is the second critical aspect of process implementation. Last week we identified the components of a process. Once they are identified and it has been determined if they are applicable then the next step is Process Improvement.

    Many organizations identify the need for Process Improvement but they seem to say it as statement sliding the two words together as if it is something that will occur without effort and frequently without results. Philip Crosby’s book “Quality is Free” talks about this and in our last post we mentioned getting the process just right. Getting it just right is not a one step process. At a minimum it is a 4 step process and those steps contain a lot of subsidiary detail.

    Initially, as mentioned last week, we have to identify the processes that are in existence. Part of NVP’s Assessment process is to identify the existing processes. This can be done internally although that can be more difficult since people tend to live through the process and cannot see it ‘from the outside’.

    Once the processes are identified, then we need to see how they can be improved. This requires thinking ‘outside the box’ and coming with fresh insight to existing processes. A critical piece of this is to determine how the suggested processes can be measured. Without measurement it is impossible to know whether improvement has taken place. Peter Drucker – “you can’t manage what you can’t measure,” and extended to say “if you can’t measure it, you can’t improve it.”

    The next step then falls into place. We take the recommended improved process and try it out while taking measurements. Once the process has had some time to be tested, the measurements are collated and it is determined if the process has improved.
    There are two possible outcomes:

    • Process Improvement has occurred. In which case then we look for further process improvement.
    • Process Improvement has not occurred. In which case we discard the suggested process and look for some other way to obtain process improvement.

    Next week we will provide some examples.

  • Quality Assurance Processes

    Quality Assurance Processes are key to a successful project. Before we get into the where and why, here is  a brief overview of processes as we see them.

    A Quality Assurance Process has several components which are often described via a workbench.
    The following are common to most of the workbenches we have used:

    • Inputs are items that are provided to the workbench. Frequently they are the output from another workbench and they should be subject to entrance criteria.
    • Some Do Process to which the Inputs are subject
    • Some Check Process after the Do Process has completed
    • Tools may be used to assist in the Do and Check Processes
    • If the product or process passes then it is delivered to the next workbench or to the final customer
    • If the product or process fails then rework is undertaken to correct the errors
    • Standards should be applied to all the other components of the workbench

    The key point is to determine where to apply the processes for the most impact and best ROI.

    Too Much Process

    There are companies (some of them no longer in business) who applied processes to absolutely every step and every thing they did. They ‘bogged down in process’ to the detriment of the actual work they were supposed to be doing. There were too many processes to remember and people actually devoted time to working out how to avoid the process.

    Too Little Process

    The opposite of the above are the companies that avoid process entirely. Everything is left to current thought of the day and it can be changed tomorrow. These companies are at Level 1 (or less) of CMMi and very little actually moves each day since everyone knows it can change before. People resort to doing as a little as possible while waiting on the final decision.

    Just Right Process

    The best way is the Just Right Processes. We will talk about that next week.

  • Validation

    Quality Assurance has a number of subdivisions; one of those being Quality Control (sometimes referred to as Software Testing). We have been discussing Verification for the last four weeks. See our blog. Today we start into Validation

    We define Validation as being the active part of testing where completed, compiled, and promoted code is being run using data and generating results while Verification is the static part of testing. Note that promoted includes Unit testing on the developer’s workstation. So the promotion may not be a very formal one nor may the compile amount to a lot.

    Validation includes All aspects of active testing including the major divisions of White Box and Black Box (along with all the various Grey boxes in between). The fact that the code has to be completed means that it is difficult to launch into Validation before a lot of work has already been done. This is what adds to the cost of completing this type of testing and ensures that any defects that are found tend to be expensive to correct and retest. Hence our emphasis on Verification as being more generally cost effective.

    A lot of work has been completed to enable Validation to proceed smoothly and effectively. Many testers start their careers in this area and then move into various aspects of the process as they become more specialized. Some go towards Automation, others move into specialised areas like Performance or Security testing.

    While the concept is simple, the identification of the most cost-effective place to use Validation techniques is not always easy. There is no point in completing Validation without some end in view and a positive ROI. Give us a call at 416-927-0960 or visit our website at NVP.ca to find out where you would benefit from the implementation of Validation techniques in your organisation.

  • Verification Applicability in Software Testing

    Last week we introduced the concept of verification and defined it. This week we want to discuss verification applicability. Many people agree that verification should be done but come to a halt when deciding where they can apply it and the degree of formality that should be used. Verification really covers everything that can be done in the project in terms of testing excluding the actual Quality Control or active testing. In other words we can verify almost everything.

    • Business Objectives – can be verified for feasibility, correctness, completeness and how accurately they reflect what the business needs
    • Requirements – can be verified for correctness, completeness, feasibility, testability, and how well they reflect the underlying business objectives
    • Design – can be verified fir correctness, completeness, feasibility, testability, and how well they represent the defining requirements
    • Code – can be verified for adherence to standards, readability , level of comments, structure, whether it reflects the design, maintainability, portability, testability
    • Test Plans – can be verified for readability, correctness, applicability to the testing at hand, ability to execute what is required, scope
    • Test Cases – can be verified for coverage, maintainability, readability, ability to execute, maintainability, portability
    • Test Data – can be verified for coverage, security (anonymity), maintainability, portability, retention
    • Test Reports – can be verified for usefullness, readability, completeness

    The above are only a partial list of the major items that can be verified. The list can include any artifact that may be produced during the project like Minutes of Meetings, Online Conversations, transcribed verbal conversations, and emails. If it can be read it can be verified!

    The issue is to find the most cost-effective places in your development process to implement verification techniques. There is no point in completing Verification processes without some end in view and a positive ROI. Give us a call at 416-927-0960 or visit our website at NVP.ca to find out where you would benefit from the implementation of Verification techniques in your organisation.

  • Examples of Verification in Software Testing

    In our last blog post, we included a list of Examples of Verification without going into too much detail. In this blog we will take a couple of those verification list items and expand on them. The examples of verification that most tend to impact software testers with the best return are Test Plans, Test Cases, Test Data and Test Results.

    Test Plans

    Applying verification techniques to a Test Plan can save hours of effort. The three methods we can use to review a test plan are:

      • Walkthrough – in this method, the author of the Test Plan ‘walks’ one or more of their peers through the test plan, explaining the sections and what is meant by the content. The role of the peers is to find problems, omissions, extra content, incomplete items, inconsistent items and to add things that they might feel were missed. The intent is to have a better document more accurately reflecting the needs of the project stakeholders. A new version is issued after all the errors are corrected and it is used going forward.
      • Document Review – In this method, a review committee (preferably selected from the interested stakeholders) reviews the documents and records the same items as listed above for the walkthrough. Once they are finished their individual reviews, they come together to create a final list of problems which need to be corrected. A new version is issued after all the errors are corrected and it is used going forward.
      • Inspections – In this method, formalized roles are defined and assigned and a procedure is followed to ensure proper inspection of the document. The intent and result is the same as the previous two methods. The only difference is the degree of formality.

    What’s the point in all of this? The payback from reviewing the plan (using any of the methods above, more than pays for itself in terms of less errors going forwrd, less work to be undone, redone and redone (again).

    If you know your test plan is poor, or you’re not even sure where to start give us a call at 416-927-0960 or visit our website at NVP.ca to find out where you would benefit from the implementation of Verification techniques in your organisation.

  • Verification

    Quality Assurance has a number of subdivisions; one of those being Quality Control (sometimes referred to as Software Testing). One of the, largely underrated, aspects of Quality Control is Verification.
    We define Verification as being the static part of testing while Validation is the active part of testing where completed, compiled, and promoted code is being run using data and generating results. Verification refers to aspects of testing that do not involve actually executing the code and includes: Inspections, Walk-throughs and Reviews.

    Since we do not need code, it is possible to launch into Verification much earlier in the process without waiting for code to be completed. As someone once pointed out, the first time you pick up some artifact from the project whether it be a document, minutes of meetings, a communication or even a conversation and start to look at it, you are performing your very first test of that system.

    The three major subdivisions mentioned above differ in their degree of formality and where they may be applied. In general Inspections are the most formal of the processes and have a well developed methodology. Reviews can be formal or informal and can be done at various points depending on how necessary they are and what is the intent. Walkthroughs can be very informal and used with minimal effort and training.

    The major impact of this type of testing is to discover issues, misconceptions, lack of clarity, incompleteness and misinterpretations as soon as possible.
    While the concept is simple, the identification of the most cost-effective place to use Verification techniques is not always easy. There is no point in completing Verification processes without some end in view and a positive ROI. Give us a call at 416-927-0960 or visit our website at NVP.ca to find out where you would benefit from the implementation of Verification techniques in your organisation.

  • Factory Testing

    Factory Testing is one the terms that leads to a lot of acrimony in the industry. There are people who believe it and will evangelize for it and others who claim it stifles every innovative thought ever made.

    Some of the people who believe in it fervently are those who have exacting expectations from their clients. They need to be able to tell them with certainty what tests were executed, when they were executed and what the results from those tests were. At the end of test, there is no question as to what has been done and the decision about whether to proceed is easy.
    Those who believe that this process stifles creativity think that simply following fixed scripts without thought reduces the role of a tester to that of a checker and not a tester and the work might as well be done by someone without a testing background.

    There is room for both views but they need to be applied at different points of the Software Development Life Cycle.

    Creativity needs to applied (in both views) at the time of Test Case or Test Objective Creation. We certainly need creativity and original thought in determining what to test and how to test it. Once that has been done and the final client has agreed that the coverage is sufficient and then the paths diverge a little, but not as much as some people think.

    In the Factory Testing method, the testers (or checkers) must finish all the tests that are approved in the way they have been written. This does not preclude them discovering other ways of testing and new tests that can and should be completed. it just means that they may not necessarily complete them immediately for the current cycle or release. They certainly should not be lost and can be used to improve the coverage and process next cycle or release or even in the current period if they are important enough.

    In the other methodologies, we can still use the approved tests as a starting point and then start to add and expand. However, we do need to know the best way of augmenting tests while ensuring coverage. Quality Assurances focus on statistical analysis will allow the calculation of those figures and make that decision.