Sometimes the most difficult thing in a Quality Assessment is the first step. While there are usually some obvious quality patterns and issues, it can be difficult to get a starting point. We always start with information gathering interviews with the client representative who requested the Assessment. Clearly they thought that there was benefit to be obtained from an Assessment and their reasoning and motivation is crucial to the subsequent success of the contract.
Blog
-
What is a Quality Assessment?
To many people, a Quality Assessment seems rather nebulous. How can one assess Quality in an organisation? A Quality Assessment looks at the Processes being used to develop or purchase and test software and compares them against industry best practices to see where improvements could be made. There are some existing standard models that can be used (TMMI for example) which provide lists of attributes and processes to compare. These can be modified to suit any particular situation and the assessment completed and mapped.
Next week, we will discuss some of the processes we follow to complete this assessment.
Lessons Learned: No one process suits every occasion and we need to be flexible in what is used to provide answers.
-
Get a Consistent Reporting Format
Reporting results is one of the bigger aspects of testing that is often overlooked in the beginning of a project. The initial efforts are aimed at getting the testing environment established and the test cases written and approved. Then the launch into testing occupies a lot of time ironing out the minor irritants to test execution. At that point the request for reports comes up and the test lead is frequently left scrambling for data to report what is requested. Failure to anticipate the reporting requirements can lead to a lot of manual work.
Lessons Learned: A few hours defining results reporting formats can save hours later in the project.
-
Get Report Mockups Early
A couple of weeks ago, we mentioned how a knowledge of report requirements early in the project would help ensure that the data existed and could be accessed. That does not solve the problem of ensuring that the reports are tested and correct. Report testing is usually late in the cycle and is squeezed for time. However, reports are critical for information and business decision making and can make or break a company. It is crucial that the reports be tested for not only appearance but for correctness. This can be very difficult if the data has been gathered in a different way.
Lessons Learned: Report testing cannot be entirely left to the end.
-
Too Many Disparate Pieces of Software (What Happens When One Fails or Goes Out of Business)
An assessment of an existing and successful system showed a lot of disparate pieces of software hooked together in various ways. An inability to properly define and build effective QA environments for testing meant that some testing was not done and taken on faith that it worked or was tested in production (and fixed). No API testing was completed so any failure of any piece of the solution had immediate repercussions throughout the product and had to be fixed on the fly.
Lessons Learned: A proper environment and knowledge and testing of the APIs would have solved a lot of the issues that showed in production.
-
Working Towards the Final Result
Finding out requirements by digging through the hierarchy endlessly or how not to implement a product. A recent upgrade included a migration from one database (home grown) to a commercially available product. It is well known that migrations are always dangerous and can be quite time consuming. The issue was that an attempt was made to replicate the existing (and mainly unknown) database into the new one rather than asking what was important to know after the migration was complete. The testing turned into a series of cycles while the testers dug deeper and deeper into the existing database.
Lessons Learned: Make sure to take a 360 degree view of the expectations before going down the testing path.
-
The Importance of User Scenarios
A couple of weeks ago, we mentioned one instance of where Edge Cases relating to monitoring of queues caused a two month slippage in the project schedule. A similar issue came up with mobile phones that the client was accustomed to use when not at the desk. The product was designed with the expectation that users would be seated at desks and controlling the product via the workstation interface. When this was not the case, a substantial set of issues came up revolving around getting the calls to the mobile phones, making sure the queues were acted on properly and reacting to new messages. A further delay was encountered working through these unanticipated changes. There was also a problem with call recording for calls starting on the mobile phones.
Lessons Learned: A scenario showing the Day-in-the-life of a user would have identified a lot of these issues.
-
The Importance of Knowing Report Requirements
Frequently reports do not come up before the end of project and obviously if the product does not work , then reporting on the results or activity will have little value until the data is correct. However, that does not preclude confirming early in the cycle that the required information is available and in a format that will allow the reports to be formatted and populated reasonably easily. A list of the existing reports (in the case of a product upgrade or a conversion) can be checked early in the project to ensure that the data is available in the new or upgraded product.
Lessons Learned: Make sure a list of the required data is available at the commencement of the project.